Jul 16 12:05:14.461659 ip-10-0-140-160 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 16 12:05:14.461674 ip-10-0-140-160 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 16 12:05:14.461683 ip-10-0-140-160 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 16 12:05:14.462134 ip-10-0-140-160 systemd[1]: Failed to start Kubernetes Kubelet. Jul 16 12:05:24.699692 ip-10-0-140-160 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 16 12:05:24.699710 ip-10-0-140-160 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 51ba8f44d76b4607bd46567c0bd2be2f -- Jul 16 12:07:30.117733 ip-10-0-140-160 systemd[1]: Starting Kubernetes Kubelet... Jul 16 12:07:30.560892 ip-10-0-140-160 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 12:07:30.560892 ip-10-0-140-160 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 12:07:30.560892 ip-10-0-140-160 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 12:07:30.560892 ip-10-0-140-160 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 12:07:30.560892 ip-10-0-140-160 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 12:07:30.560892 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.517379 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 12:07:30.560892 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524527 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:07:30.560892 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524545 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:07:30.560892 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524551 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:07:30.560892 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524555 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:07:30.560892 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524564 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:07:30.560892 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524568 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:07:30.560892 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524573 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:07:30.560892 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524577 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524588 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524593 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524598 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524602 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524606 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524620 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524624 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524629 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524633 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524637 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524645 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524650 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524654 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524658 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524661 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524668 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524672 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524676 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524680 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:07:30.562449 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524684 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524688 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524693 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524696 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524700 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524704 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524710 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524715 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524719 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524722 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524726 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524730 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524738 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524742 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524746 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524751 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524754 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524758 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524765 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524769 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:07:30.563197 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524774 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524778 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524781 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524784 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524793 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524798 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524801 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524805 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524809 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524812 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524820 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524824 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524828 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524833 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524838 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524846 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524850 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524854 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524858 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:07:30.563992 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524862 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524866 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524874 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524878 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524882 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524886 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524890 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524899 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524903 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524907 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524911 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524915 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524918 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524921 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524929 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524932 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524936 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524940 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524943 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524946 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:07:30.564774 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524956 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524961 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524966 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524970 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524974 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524982 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524986 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524991 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524994 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.524998 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.525001 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.525009 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.525012 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526294 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526305 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526314 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526319 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526324 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526333 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526343 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:07:30.566341 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526347 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526352 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526356 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526360 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526368 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526372 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526376 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526380 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526384 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526388 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526391 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526399 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526403 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526407 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526429 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526433 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526437 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526441 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526444 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526453 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:07:30.567590 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526457 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526461 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526465 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526468 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526472 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526479 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526483 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526486 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526490 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526494 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526498 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526506 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526510 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526515 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526518 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526522 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526526 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526533 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526538 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:07:30.568786 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526547 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526551 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526556 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526559 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526563 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526570 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526574 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526578 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526585 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526590 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526600 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526604 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526609 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526613 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526616 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526625 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526629 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526633 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526637 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526641 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:07:30.570011 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526645 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526654 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526660 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526666 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526669 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526678 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526682 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526686 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526690 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526694 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526697 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526701 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526709 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526714 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526718 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526721 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526725 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526729 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526737 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:07:30.575864 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526740 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526748 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526752 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526760 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526766 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526770 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526773 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526777 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526781 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526789 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526793 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526798 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526802 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.526806 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527465 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527478 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527482 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527486 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527491 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527499 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:07:30.576977 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527503 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527507 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527511 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527515 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527527 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527533 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527537 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527541 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527545 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527552 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527556 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527560 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527564 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527573 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527579 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527583 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527586 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527590 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527594 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527602 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:07:30.577737 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527606 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527609 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527613 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527616 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527620 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527627 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527632 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527636 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527640 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527643 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527647 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527655 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527659 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527663 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527667 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527676 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527681 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527685 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527689 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527693 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:07:30.579310 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527697 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527704 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527708 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527712 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527716 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527721 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527724 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527732 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527736 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527740 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527746 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527750 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527758 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527763 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527767 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527771 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527775 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527783 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527788 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:07:30.580026 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527792 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527796 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527799 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527803 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527811 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527815 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527819 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527824 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527827 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527837 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527841 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527845 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527849 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527853 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527857 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527866 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527871 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527874 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527880 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:07:30.580803 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527883 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527887 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527890 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527894 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527897 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527905 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527908 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527912 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527915 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527919 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527923 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527930 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527934 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527938 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.527942 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528060 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528070 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528081 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528087 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528093 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528098 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 12:07:30.581960 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528108 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528117 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528122 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528126 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528135 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528140 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528145 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528149 2578 flags.go:64] FLAG: --cgroup-root="" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528153 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528161 2578 flags.go:64] FLAG: --client-ca-file="" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528167 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528172 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528183 2578 flags.go:64] FLAG: --cluster-domain="" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528188 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528193 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528197 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528202 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528211 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528215 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528220 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528224 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528228 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528237 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528242 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 12:07:30.583187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528246 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528251 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528256 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528265 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528270 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528274 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528279 2578 flags.go:64] FLAG: --enable-server="true" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528284 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528293 2578 flags.go:64] FLAG: --event-burst="100" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528299 2578 flags.go:64] FLAG: --event-qps="50" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528304 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528308 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528316 2578 flags.go:64] FLAG: --eviction-hard="" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528322 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528326 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528330 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528334 2578 flags.go:64] FLAG: --eviction-soft="" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528338 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528344 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528349 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528356 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528360 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528365 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528369 2578 flags.go:64] FLAG: --feature-gates="" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528374 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 12:07:30.584378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528383 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528388 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528392 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528397 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528401 2578 flags.go:64] FLAG: --help="false" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528422 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-140-160.ec2.internal" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528432 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528437 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528441 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528446 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528451 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528459 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528463 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528467 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528472 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528484 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528491 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528496 2578 flags.go:64] FLAG: --kube-reserved="" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528500 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528504 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528512 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528516 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528521 2578 flags.go:64] FLAG: --lock-file="" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528525 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 12:07:30.585120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528529 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528540 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528547 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528552 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528556 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528560 2578 flags.go:64] FLAG: --logging-format="text" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528568 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528573 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528578 2578 flags.go:64] FLAG: --manifest-url="" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528582 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528588 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528597 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528603 2578 flags.go:64] FLAG: --max-pods="110" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528607 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528633 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528643 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528649 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528654 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528658 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528663 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528679 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528683 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528688 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 12:07:30.586736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528697 2578 flags.go:64] FLAG: --pod-cidr="" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528704 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528712 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528716 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528725 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528729 2578 flags.go:64] FLAG: --port="10250" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528734 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528739 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b431c1e49e40b7ef" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528743 2578 flags.go:64] FLAG: --qos-reserved="" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528747 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528757 2578 flags.go:64] FLAG: --register-node="true" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528761 2578 flags.go:64] FLAG: --register-with-taints="" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528767 2578 flags.go:64] FLAG: --registry-burst="10" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528771 2578 flags.go:64] FLAG: --registry-qps="5" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528779 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528783 2578 flags.go:64] FLAG: --reserved-memory="" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528788 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528793 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528797 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528801 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528805 2578 flags.go:64] FLAG: --runonce="false" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528809 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528817 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528821 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528826 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 12:07:30.587899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528830 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528834 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528842 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528847 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528851 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528855 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528859 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528867 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528871 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528878 2578 flags.go:64] FLAG: --system-cgroups="" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528886 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528893 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528898 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528901 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528908 2578 flags.go:64] FLAG: --tls-min-version="" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528916 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528920 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528926 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528930 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528934 2578 flags.go:64] FLAG: --v="2" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528944 2578 flags.go:64] FLAG: --version="false" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528950 2578 flags.go:64] FLAG: --vmodule="" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528955 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.528959 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 12:07:30.589466 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529150 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529158 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529163 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529172 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529177 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529181 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529185 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529188 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529196 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529200 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529204 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529208 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529211 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529215 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529222 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529226 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529230 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529235 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529239 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529243 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:07:30.590233 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529247 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529250 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529254 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529262 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529265 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529272 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529275 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529279 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529287 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529291 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529296 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529299 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529303 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529306 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529314 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529317 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529321 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529325 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529329 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529332 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:07:30.591498 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529340 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529344 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529348 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529351 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529355 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529358 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529362 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529370 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529374 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529379 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529383 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529387 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529395 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529399 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529403 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529407 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529427 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529433 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529437 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529445 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:07:30.592247 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529449 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529452 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529459 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529463 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529472 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529476 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529480 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529484 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529487 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529491 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529494 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529502 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529506 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529510 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529514 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529517 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529521 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529529 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529532 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:07:30.593371 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529536 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529540 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529546 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529549 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529557 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529562 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529566 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529569 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529573 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529580 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529586 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529590 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529595 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529599 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529608 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529612 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529616 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529620 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529624 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:07:30.594231 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.529627 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:07:30.595792 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.529639 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 12:07:30.595792 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.536879 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 12:07:30.595792 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.536894 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 12:07:30.595792 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.536910 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 12:07:30.595792 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.536923 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 12:07:30.595792 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.536975 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:07:30.595792 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.536979 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:07:30.595792 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.536983 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:07:30.595792 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.536986 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:07:30.595792 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.536989 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:07:30.595792 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.536992 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:07:30.595792 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.536994 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:07:30.595792 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.536997 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:07:30.595792 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.536999 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:07:30.595792 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537002 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537004 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537007 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537010 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537013 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537016 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537018 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537021 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537023 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537025 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537028 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537031 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537033 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537036 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537038 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537041 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537044 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537047 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537049 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537053 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:07:30.596560 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537055 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537058 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537060 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537062 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537066 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537070 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537073 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537076 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537079 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537081 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537084 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537087 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537090 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537092 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537095 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537098 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537101 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537103 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537106 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:07:30.599993 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537109 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:07:30.598364 ip-10-0-140-160 systemd[1]: Started Kubernetes Kubelet. Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537111 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537113 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537116 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537119 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537122 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537124 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537127 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537129 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537132 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537135 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537137 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537140 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537142 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537145 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537147 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537150 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537152 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537154 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537157 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:07:30.601296 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537159 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537162 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537164 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537167 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537169 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537172 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537175 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537178 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537181 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537184 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537187 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537190 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537193 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537195 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537199 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537206 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537209 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537212 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537214 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:07:30.707321 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537217 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:07:30.708263 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537219 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:07:30.708263 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537227 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:07:30.708263 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537229 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:07:30.708263 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537232 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:07:30.708263 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537235 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:07:30.708263 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537237 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:07:30.708263 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537239 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:07:30.708263 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537242 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:07:30.708263 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537245 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:07:30.708263 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537247 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:07:30.708263 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537249 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:07:30.708263 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.537254 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 12:07:30.708263 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537361 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:07:30.708263 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537366 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:07:30.708263 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537369 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537371 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537374 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537376 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537379 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537381 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537384 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537386 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537388 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537391 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537394 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537396 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537398 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537401 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537405 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537423 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537429 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537432 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537434 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:07:30.709045 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537438 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537442 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537445 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537448 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537451 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537453 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537456 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537459 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537461 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537464 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537467 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537469 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537472 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537474 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537476 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537479 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537482 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537485 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537488 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537491 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:07:30.709857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537493 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537496 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537498 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537502 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537504 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537506 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537509 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537513 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537515 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537518 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537520 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537523 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537526 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537529 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537531 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537534 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537537 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537539 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537542 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537544 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:07:30.713191 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537547 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537549 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537552 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537554 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537557 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537559 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537562 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537564 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537567 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537569 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537571 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537574 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537576 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537579 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537581 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537584 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537589 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537591 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537594 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:07:30.714427 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537597 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537600 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537602 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537605 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537607 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537611 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537614 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537617 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537619 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537622 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537625 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537627 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537630 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537632 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537651 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537655 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537658 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537661 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:07:30.716174 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:30.537664 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:07:30.716987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.537669 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 12:07:30.716987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.537767 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 12:07:30.716987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.539887 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 12:07:30.716987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.541262 2578 server.go:1019] "Starting client certificate rotation" Jul 16 12:07:30.716987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.541358 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 12:07:30.716987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.541395 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 12:07:30.716987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.543888 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 12:07:30.716987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.545364 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 12:07:30.716987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.556386 2578 log.go:25] "Validated CRI v1 runtime API" Jul 16 12:07:30.716987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.559057 2578 log.go:25] "Validated CRI v1 image API" Jul 16 12:07:30.716987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.560034 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 12:07:30.716987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.564990 2578 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 bc1306e4-1758-43fc-a1d8-018ef11014a5:/dev/nvme0n1p3 f54f3e36-aa25-4234-954c-c4b51cdc8409:/dev/nvme0n1p4] Jul 16 12:07:30.730399 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.565013 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 12:07:30.730797 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.571594 2578 manager.go:216] Machine: {Timestamp:2026-07-16 12:07:30.569979467 +0000 UTC m=+0.333348925 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3201626 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f01c6c0915a0d0247a06179d0c095 SystemUUID:ec2f01c6-c091-5a0d-0247-a06179d0c095 BootID:51ba8f44-d76b-4607-bd46-567c0bd2be2f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:af:e6:2c:05:93 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:af:e6:2c:05:93 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3e:f1:04:84:38:4b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 12:07:30.730797 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.571723 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 12:07:30.730797 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.571881 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 12:07:30.730797 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.575331 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 12:07:30.730797 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.575377 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-160.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 12:07:30.730797 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.576157 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 12:07:30.730797 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.576173 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 12:07:30.730797 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.576185 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.577438 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.578752 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.581778 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.581906 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.581942 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.581954 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.581971 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.581982 2578 kubelet.go:398] "Adding apiserver pod source" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.581994 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.582488 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.582522 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.585825 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.586451 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.587696 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.587723 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.587917 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.587942 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.587954 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.587962 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.587973 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.587981 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.589192 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.589201 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.589246 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.589257 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.589268 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.589281 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.590214 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.590226 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.593372 2578 server.go:1268] "Started kubelet" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.593486 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.593584 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.593659 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.599642 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.600624 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.602329 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.604729 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.604740 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.604836 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-160.ec2.internal\" not found" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.605652 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.605754 2578 factory.go:55] Registering systemd factory Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.605770 2578 factory.go:223] Registration of the systemd container factory successfully Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.605867 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.605883 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 12:07:30.732452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.605987 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.605998 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.606053 2578 factory.go:153] Registering CRI-O factory Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.606067 2578 factory.go:223] Registration of the crio container factory successfully Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.606121 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.606147 2578 factory.go:103] Registering Raw factory Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.606161 2578 manager.go:1196] Started watching for new ooms in manager Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.606268 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.606565 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.606681 2578 manager.go:349] Starting recovery of all containers Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.605748 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-160.ec2.internal.18c2c3c4609254d7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-160.ec2.internal,UID:ip-10-0-140-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-160.ec2.internal,},FirstTimestamp:2026-07-16 12:07:30.593346775 +0000 UTC m=+0.356716229,LastTimestamp:2026-07-16 12:07:30.593346775 +0000 UTC m=+0.356716229,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-160.ec2.internal,}" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.610804 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-160.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.618986 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.619891 2578 manager.go:354] Recovery completed Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.621597 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.621625 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.626001 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.628241 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.628285 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.628302 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.629009 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.629020 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.629023 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.629039 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.629881 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.629918 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.629931 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.630463 2578 policy_none.go:49] "None policy: Start" Jul 16 12:07:30.734131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.630479 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 12:07:30.736614 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.630508 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 12:07:30.736614 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.630952 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-160.ec2.internal.18c2c3c462a719f4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-160.ec2.internal,UID:ip-10-0-140-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-160.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-160.ec2.internal,},FirstTimestamp:2026-07-16 12:07:30.628262388 +0000 UTC m=+0.391631849,LastTimestamp:2026-07-16 12:07:30.628262388 +0000 UTC m=+0.391631849,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-160.ec2.internal,}" Jul 16 12:07:30.736614 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.631713 2578 policy_none.go:47] "Start" Jul 16 12:07:30.736614 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.645264 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-160.ec2.internal.18c2c3c462a791e1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-160.ec2.internal,UID:ip-10-0-140-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-160.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-160.ec2.internal,},FirstTimestamp:2026-07-16 12:07:30.628293089 +0000 UTC m=+0.391662553,LastTimestamp:2026-07-16 12:07:30.628293089 +0000 UTC m=+0.391662553,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-160.ec2.internal,}" Jul 16 12:07:30.736614 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.656776 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-160.ec2.internal.18c2c3c462a7dfe6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-160.ec2.internal,UID:ip-10-0-140-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-160.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-160.ec2.internal,},FirstTimestamp:2026-07-16 12:07:30.628313062 +0000 UTC m=+0.391682520,LastTimestamp:2026-07-16 12:07:30.628313062 +0000 UTC m=+0.391682520,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-160.ec2.internal,}" Jul 16 12:07:30.736614 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.672070 2578 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-160.ec2.internal.18c2c3c462a719f4\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-160.ec2.internal.18c2c3c462a719f4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-160.ec2.internal,UID:ip-10-0-140-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-160.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-160.ec2.internal,},FirstTimestamp:2026-07-16 12:07:30.628262388 +0000 UTC m=+0.391631849,LastTimestamp:2026-07-16 12:07:30.629903767 +0000 UTC m=+0.393273224,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-160.ec2.internal,}" Jul 16 12:07:30.736614 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.684051 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-db6mb" Jul 16 12:07:30.737045 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.685390 2578 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-160.ec2.internal.18c2c3c462a791e1\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-160.ec2.internal.18c2c3c462a791e1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-160.ec2.internal,UID:ip-10-0-140-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-160.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-160.ec2.internal,},FirstTimestamp:2026-07-16 12:07:30.628293089 +0000 UTC m=+0.391662553,LastTimestamp:2026-07-16 12:07:30.629923585 +0000 UTC m=+0.393293042,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-160.ec2.internal,}" Jul 16 12:07:30.737045 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.700195 2578 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-160.ec2.internal.18c2c3c462a7dfe6\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-160.ec2.internal.18c2c3c462a7dfe6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-160.ec2.internal,UID:ip-10-0-140-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-160.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-160.ec2.internal,},FirstTimestamp:2026-07-16 12:07:30.628313062 +0000 UTC m=+0.391682520,LastTimestamp:2026-07-16 12:07:30.629935999 +0000 UTC m=+0.393305459,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-160.ec2.internal,}" Jul 16 12:07:30.737045 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:30.702585 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-db6mb" Jul 16 12:07:30.737045 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.705538 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-160.ec2.internal\" not found" Jul 16 12:07:30.847353 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.806492 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-160.ec2.internal\" not found" Jul 16 12:07:30.847353 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.839725 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms" Jul 16 12:07:30.975509 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:30.907125 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-160.ec2.internal\" not found" Jul 16 12:07:31.099984 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:31.007714 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-160.ec2.internal\" not found" Jul 16 12:07:31.212454 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:31.107848 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-160.ec2.internal\" not found" Jul 16 12:07:31.212454 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:31.208701 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-160.ec2.internal\" not found" Jul 16 12:07:31.270948 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:31.270921 2578 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-160.ec2.internal\" not found" node="ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.308917 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:31.308894 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-160.ec2.internal\" not found" Jul 16 12:07:31.335313 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.335143 2578 manager.go:337] "Starting Device Plugin manager" Jul 16 12:07:31.335434 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:31.335354 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 12:07:31.335434 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.335367 2578 server.go:87] "Starting device plugin registration server" Jul 16 12:07:31.335627 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.335615 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 12:07:31.335664 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.335629 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 12:07:31.335913 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.335728 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 12:07:31.335913 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.335829 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 12:07:31.335913 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.335839 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 12:07:31.336455 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:31.336428 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 12:07:31.336540 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:31.336526 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-160.ec2.internal\" not found" Jul 16 12:07:31.436778 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.436737 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:07:31.438017 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.437780 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:07:31.438017 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.437806 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:07:31.438017 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.437822 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:07:31.438017 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.437855 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.454292 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.454272 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.454359 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:31.454295 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-160.ec2.internal\": node \"ip-10-0-140-160.ec2.internal\" not found" Jul 16 12:07:31.472692 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.472666 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 12:07:31.472799 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.472701 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 12:07:31.472799 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.472729 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 12:07:31.472799 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:31.472784 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 12:07:31.479148 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.479126 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 12:07:31.497147 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:31.497123 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-160.ec2.internal\" not found" Jul 16 12:07:31.540642 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.540612 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 12:07:31.540793 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.540746 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 12:07:31.574419 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.574386 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-160.ec2.internal"] Jul 16 12:07:31.574823 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.574488 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:07:31.576912 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.576895 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:07:31.576999 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.576926 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:07:31.576999 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.576937 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:07:31.578331 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.578317 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:07:31.578482 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.578465 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.578573 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.578518 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:07:31.579166 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.579150 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:07:31.579260 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.579168 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:07:31.579260 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.579196 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:07:31.579260 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.579211 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:07:31.579260 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.579177 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:07:31.579466 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.579268 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:07:31.580599 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.580581 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.580680 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.580611 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:07:31.581357 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.581336 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:07:31.581453 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.581364 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:07:31.581453 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.581374 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:07:31.592928 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.592913 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 12:07:31.605127 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.605107 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.605127 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.605124 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 12:07:31.605259 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.605226 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.608703 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.608688 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.612912 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.612895 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/05e2cca6f5dc03b998a60a299686fd5b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal\" (UID: \"05e2cca6f5dc03b998a60a299686fd5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.612998 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.612933 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05e2cca6f5dc03b998a60a299686fd5b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal\" (UID: \"05e2cca6f5dc03b998a60a299686fd5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.612998 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.612959 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4e3b2dbb5e31a37252e72dba16a64d8f-config\") pod \"kube-apiserver-proxy-ip-10-0-140-160.ec2.internal\" (UID: \"4e3b2dbb5e31a37252e72dba16a64d8f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.641361 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.641339 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 12:07:31.711337 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.711264 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 12:07:31.713360 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.713344 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4e3b2dbb5e31a37252e72dba16a64d8f-config\") pod \"kube-apiserver-proxy-ip-10-0-140-160.ec2.internal\" (UID: \"4e3b2dbb5e31a37252e72dba16a64d8f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.713414 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.713368 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/05e2cca6f5dc03b998a60a299686fd5b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal\" (UID: \"05e2cca6f5dc03b998a60a299686fd5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.713414 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.713393 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05e2cca6f5dc03b998a60a299686fd5b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal\" (UID: \"05e2cca6f5dc03b998a60a299686fd5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.713502 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.713458 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05e2cca6f5dc03b998a60a299686fd5b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal\" (UID: \"05e2cca6f5dc03b998a60a299686fd5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.713502 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.713465 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4e3b2dbb5e31a37252e72dba16a64d8f-config\") pod \"kube-apiserver-proxy-ip-10-0-140-160.ec2.internal\" (UID: \"4e3b2dbb5e31a37252e72dba16a64d8f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.713502 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.713476 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/05e2cca6f5dc03b998a60a299686fd5b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal\" (UID: \"05e2cca6f5dc03b998a60a299686fd5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.722064 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.722049 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 12:07:31.722923 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.722909 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.723944 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.723932 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 12:07:31.725501 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.725487 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.736995 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.736974 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 12:07:31.737102 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:31.737022 2578 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-160.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.739388 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.739369 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.798273 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.798249 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 12:07:31.798419 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:31.798296 2578 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal" Jul 16 12:07:31.821848 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.821821 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cxx5m" Jul 16 12:07:31.870552 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:31.870531 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cxx5m" Jul 16 12:07:32.011979 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.011900 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 12:07:32.166434 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.166359 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 12:02:31 +0000 UTC" deadline="2028-01-03 09:15:45.214833379 +0000 UTC" Jul 16 12:07:32.166434 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.166412 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12861h8m13.048425232s" Jul 16 12:07:32.585464 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.585392 2578 apiserver.go:52] "Watching apiserver" Jul 16 12:07:32.590907 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.590884 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 12:07:32.591311 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.591290 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc","openshift-cluster-node-tuning-operator/tuned-9gd89","openshift-dns/node-resolver-r7c42","openshift-image-registry/node-ca-wjpmx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal","openshift-network-diagnostics/network-check-target-8lp25","openshift-ovn-kubernetes/ovnkube-node-cx284","kube-system/global-pull-secret-syncer-mq859","kube-system/kube-apiserver-proxy-ip-10-0-140-160.ec2.internal","openshift-multus/multus-additional-cni-plugins-hcnsq","openshift-multus/multus-ct6tl","openshift-multus/network-metrics-daemon-rdpjz","openshift-network-operator/iptables-alerter-ss4rt","kube-system/konnectivity-agent-9p5kr"] Jul 16 12:07:32.592810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.592794 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.593930 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.593908 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.595069 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.595052 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r7c42" Jul 16 12:07:32.596407 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.596393 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wjpmx" Jul 16 12:07:32.597561 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.597546 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:32.597628 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:32.597609 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:07:32.598940 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.598927 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.600153 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.600134 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:32.600243 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:32.600200 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:07:32.601588 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.601571 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.602833 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.602811 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 12:07:32.602963 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.602945 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.604712 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.604694 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:32.604800 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:32.604780 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:07:32.606271 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.606256 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ss4rt" Jul 16 12:07:32.607527 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.607509 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9p5kr" Jul 16 12:07:32.619730 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.619703 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-run\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.619831 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.619738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jv59\" (UniqueName: \"kubernetes.io/projected/8bdebfdd-8e6f-4154-8d24-9e0c06229293-kube-api-access-4jv59\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.619831 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.619764 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-cni-binary-copy\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.619831 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.619798 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret\") pod \"global-pull-secret-syncer-mq859\" (UID: \"a0853c65-0aa1-4d5f-aab4-345f5d30f85f\") " pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:32.619970 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.619847 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-system-cni-dir\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.619970 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.619924 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-system-cni-dir\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.619970 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.619944 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-run-netns\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.619970 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.619963 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/833615c3-9d95-448d-a709-4735c575a995-serviceca\") pod \"node-ca-wjpmx\" (UID: \"833615c3-9d95-448d-a709-4735c575a995\") " pod="openshift-image-registry/node-ca-wjpmx" Jul 16 12:07:32.620121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.619985 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-host\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.620121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620023 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-var-lib-cni-bin\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.620121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620042 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-hostroot\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.620121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620078 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-multus-conf-dir\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.620121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620099 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/374df003-cae9-437a-a50e-2928d6a8b43d-tmp\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.620121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620114 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj8tv\" (UniqueName: \"kubernetes.io/projected/374df003-cae9-437a-a50e-2928d6a8b43d-kube-api-access-nj8tv\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.620406 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620130 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-run-systemd\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.620406 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620169 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a75d700b-84c1-48b2-85bb-dd190d3fd093-cni-binary-copy\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.620406 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620210 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-etc-kubernetes\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.620406 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620243 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.620406 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620270 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-sysctl-d\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.620406 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620297 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-var-lib-kubelet\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.620406 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620321 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e00ecaab-22f9-4338-ac72-f0eaad408347-hosts-file\") pod \"node-resolver-r7c42\" (UID: \"e00ecaab-22f9-4338-ac72-f0eaad408347\") " pod="openshift-dns/node-resolver-r7c42" Jul 16 12:07:32.620406 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620345 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-node-log\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.620406 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620368 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-log-socket\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.620406 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620398 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/82627060-8f4c-4020-8e86-506df37a1e55-ovnkube-config\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.620774 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620422 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-cnibin\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.620774 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620469 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzp6v\" (UniqueName: \"kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v\") pod \"network-check-target-8lp25\" (UID: \"2d5d88f9-0022-4b17-9d13-8fcbc76ef75d\") " pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:32.620774 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620493 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-multus-socket-dir-parent\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.620774 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620517 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-var-lib-cni-multus\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.620774 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620542 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-var-lib-kubelet\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.620774 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620582 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a75d700b-84c1-48b2-85bb-dd190d3fd093-multus-daemon-config\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.620774 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620609 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-run-multus-certs\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.620774 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620626 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpdkr\" (UniqueName: \"kubernetes.io/projected/6f3b3c2d-0128-4354-aa5d-014078c0cd04-kube-api-access-lpdkr\") pod \"iptables-alerter-ss4rt\" (UID: \"6f3b3c2d-0128-4354-aa5d-014078c0cd04\") " pod="openshift-network-operator/iptables-alerter-ss4rt" Jul 16 12:07:32.620774 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620648 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-socket-dir\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.620774 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620677 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-device-dir\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.620774 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620706 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.620774 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620733 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdqrw\" (UniqueName: \"kubernetes.io/projected/e00ecaab-22f9-4338-ac72-f0eaad408347-kube-api-access-mdqrw\") pod \"node-resolver-r7c42\" (UID: \"e00ecaab-22f9-4338-ac72-f0eaad408347\") " pod="openshift-dns/node-resolver-r7c42" Jul 16 12:07:32.620774 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620762 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-var-lib-openvswitch\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.621150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620785 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/82627060-8f4c-4020-8e86-506df37a1e55-ovn-node-metrics-cert\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.621150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620802 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-multus-cni-dir\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.621150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620827 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-os-release\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.621150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620860 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-run-k8s-cni-cncf-io\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.621150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620878 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f3b3c2d-0128-4354-aa5d-014078c0cd04-host-slash\") pod \"iptables-alerter-ss4rt\" (UID: \"6f3b3c2d-0128-4354-aa5d-014078c0cd04\") " pod="openshift-network-operator/iptables-alerter-ss4rt" Jul 16 12:07:32.621150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620910 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-etc-openvswitch\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.621150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620952 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-sysctl-conf\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.621150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.620985 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-etc-selinux\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.621150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621006 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-sys-fs\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.621150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-lib-modules\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.621150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621046 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs\") pod \"network-metrics-daemon-rdpjz\" (UID: \"e73a1e88-85a5-4b47-bf8f-372eec90eafd\") " pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:32.621150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621067 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6f3b3c2d-0128-4354-aa5d-014078c0cd04-iptables-alerter-script\") pod \"iptables-alerter-ss4rt\" (UID: \"6f3b3c2d-0128-4354-aa5d-014078c0cd04\") " pod="openshift-network-operator/iptables-alerter-ss4rt" Jul 16 12:07:32.621150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621092 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-sysconfig\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.621150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621113 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-sys\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.621635 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621132 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e00ecaab-22f9-4338-ac72-f0eaad408347-tmp-dir\") pod \"node-resolver-r7c42\" (UID: \"e00ecaab-22f9-4338-ac72-f0eaad408347\") " pod="openshift-dns/node-resolver-r7c42" Jul 16 12:07:32.621635 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621157 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-kubelet\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.621635 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621180 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-cni-netd\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.621635 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621198 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/82627060-8f4c-4020-8e86-506df37a1e55-ovnkube-script-lib\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.621635 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621223 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmbh4\" (UniqueName: \"kubernetes.io/projected/a75d700b-84c1-48b2-85bb-dd190d3fd093-kube-api-access-tmbh4\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.621635 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621243 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-os-release\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.621635 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621266 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-modprobe-d\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.621635 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621293 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-kubernetes\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.621635 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621308 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-slash\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.621635 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621322 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-kubelet-config\") pod \"global-pull-secret-syncer-mq859\" (UID: \"a0853c65-0aa1-4d5f-aab4-345f5d30f85f\") " pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:32.621635 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621337 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.621635 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621358 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-run-openvswitch\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.621635 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621378 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b8hv\" (UniqueName: \"kubernetes.io/projected/82627060-8f4c-4020-8e86-506df37a1e55-kube-api-access-8b8hv\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.621635 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621391 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-dbus\") pod \"global-pull-secret-syncer-mq859\" (UID: \"a0853c65-0aa1-4d5f-aab4-345f5d30f85f\") " pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:32.622114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621405 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.622114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621435 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwdw9\" (UniqueName: \"kubernetes.io/projected/e73a1e88-85a5-4b47-bf8f-372eec90eafd-kube-api-access-jwdw9\") pod \"network-metrics-daemon-rdpjz\" (UID: \"e73a1e88-85a5-4b47-bf8f-372eec90eafd\") " pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:32.622114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621457 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-run-ovn-kubernetes\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.622114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621477 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbn8j\" (UniqueName: \"kubernetes.io/projected/833615c3-9d95-448d-a709-4735c575a995-kube-api-access-zbn8j\") pod \"node-ca-wjpmx\" (UID: \"833615c3-9d95-448d-a709-4735c575a995\") " pod="openshift-image-registry/node-ca-wjpmx" Jul 16 12:07:32.622114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621504 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/374df003-cae9-437a-a50e-2928d6a8b43d-etc-tuned\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.622114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621519 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-run-netns\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.622114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621561 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-cni-bin\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.622114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621583 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-registration-dir\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.622114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621603 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/833615c3-9d95-448d-a709-4735c575a995-host\") pod \"node-ca-wjpmx\" (UID: \"833615c3-9d95-448d-a709-4735c575a995\") " pod="openshift-image-registry/node-ca-wjpmx" Jul 16 12:07:32.622114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621625 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-cnibin\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.622114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621642 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdts7\" (UniqueName: \"kubernetes.io/projected/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-kube-api-access-sdts7\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.622114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621697 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-systemd\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.622114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621737 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-systemd-units\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.622514 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621771 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-run-ovn\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.622514 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621796 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.622514 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.621820 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/82627060-8f4c-4020-8e86-506df37a1e55-env-overrides\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.625269 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.625205 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 12:07:32.625372 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.625295 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 12:07:32.625372 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.625315 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 12:07:32.625372 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.625313 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 12:07:32.625558 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.625375 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 12:07:32.625898 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.625842 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 12:07:32.626121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.626102 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 12:07:32.626313 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.626288 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 12:07:32.626536 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.626480 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 12:07:32.626610 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.626569 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 12:07:32.626610 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.626586 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 12:07:32.626766 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.626746 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 12:07:32.626865 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.626795 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 12:07:32.626865 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.626820 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vdv8q\"" Jul 16 12:07:32.627009 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.626987 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 12:07:32.627070 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.627008 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 12:07:32.627070 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.627052 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 12:07:32.627190 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.627119 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 12:07:32.627190 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.627155 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 12:07:32.627314 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.627297 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 12:07:32.627423 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.627405 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jkqj9\"" Jul 16 12:07:32.627700 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.627511 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dbrpz\"" Jul 16 12:07:32.627700 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.627541 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 12:07:32.627700 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.627570 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 12:07:32.628929 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.627668 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-66vtx\"" Jul 16 12:07:32.628929 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.628102 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hzn77\"" Jul 16 12:07:32.628929 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.628215 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fggxn\"" Jul 16 12:07:32.628929 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.628265 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 12:07:32.628929 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.628437 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bxc5v\"" Jul 16 12:07:32.629347 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.629312 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 12:07:32.629347 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.629345 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 12:07:32.629567 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.629403 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 12:07:32.629668 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.629651 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-29t4l\"" Jul 16 12:07:32.629742 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.629689 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 12:07:32.630776 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.630750 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fc227\"" Jul 16 12:07:32.706732 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.706714 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 12:07:32.722035 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722014 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-sysconfig\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.722135 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722043 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-sys\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.722135 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722063 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e00ecaab-22f9-4338-ac72-f0eaad408347-tmp-dir\") pod \"node-resolver-r7c42\" (UID: \"e00ecaab-22f9-4338-ac72-f0eaad408347\") " pod="openshift-dns/node-resolver-r7c42" Jul 16 12:07:32.722135 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722079 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-kubelet\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.722135 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-cni-netd\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.722322 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722156 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-cni-netd\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.722322 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722166 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-sys\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.722322 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722197 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-kubelet\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.722322 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722210 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/82627060-8f4c-4020-8e86-506df37a1e55-ovnkube-script-lib\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.722322 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722232 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmbh4\" (UniqueName: \"kubernetes.io/projected/a75d700b-84c1-48b2-85bb-dd190d3fd093-kube-api-access-tmbh4\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.722322 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722251 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-os-release\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.722322 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722234 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-sysconfig\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.722322 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722269 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-modprobe-d\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.722322 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722295 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-kubernetes\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.722322 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722315 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-slash\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.722890 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722331 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-kubelet-config\") pod \"global-pull-secret-syncer-mq859\" (UID: \"a0853c65-0aa1-4d5f-aab4-345f5d30f85f\") " pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:32.722890 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722333 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-os-release\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.722890 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722350 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.722890 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722377 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-run-openvswitch\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.722890 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722392 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-kubernetes\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.722890 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b8hv\" (UniqueName: \"kubernetes.io/projected/82627060-8f4c-4020-8e86-506df37a1e55-kube-api-access-8b8hv\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.722890 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722426 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-dbus\") pod \"global-pull-secret-syncer-mq859\" (UID: \"a0853c65-0aa1-4d5f-aab4-345f5d30f85f\") " pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:32.722890 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722545 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-modprobe-d\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.722890 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722618 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-run-openvswitch\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.722890 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722621 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.722890 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722638 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-kubelet-config\") pod \"global-pull-secret-syncer-mq859\" (UID: \"a0853c65-0aa1-4d5f-aab4-345f5d30f85f\") " pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:32.722890 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722649 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e00ecaab-22f9-4338-ac72-f0eaad408347-tmp-dir\") pod \"node-resolver-r7c42\" (UID: \"e00ecaab-22f9-4338-ac72-f0eaad408347\") " pod="openshift-dns/node-resolver-r7c42" Jul 16 12:07:32.722890 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722451 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.722890 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722709 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwdw9\" (UniqueName: \"kubernetes.io/projected/e73a1e88-85a5-4b47-bf8f-372eec90eafd-kube-api-access-jwdw9\") pod \"network-metrics-daemon-rdpjz\" (UID: \"e73a1e88-85a5-4b47-bf8f-372eec90eafd\") " pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:32.723566 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722739 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6fc5ae7e-11cb-4716-8341-d6613923b14b-konnectivity-ca\") pod \"konnectivity-agent-9p5kr\" (UID: \"6fc5ae7e-11cb-4716-8341-d6613923b14b\") " pod="kube-system/konnectivity-agent-9p5kr" Jul 16 12:07:32.723566 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722748 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.723566 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722761 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-run-ovn-kubernetes\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.723566 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722790 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbn8j\" (UniqueName: \"kubernetes.io/projected/833615c3-9d95-448d-a709-4735c575a995-kube-api-access-zbn8j\") pod \"node-ca-wjpmx\" (UID: \"833615c3-9d95-448d-a709-4735c575a995\") " pod="openshift-image-registry/node-ca-wjpmx" Jul 16 12:07:32.723566 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722797 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-dbus\") pod \"global-pull-secret-syncer-mq859\" (UID: \"a0853c65-0aa1-4d5f-aab4-345f5d30f85f\") " pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:32.723566 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722811 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-run-ovn-kubernetes\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.723566 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722817 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/374df003-cae9-437a-a50e-2928d6a8b43d-etc-tuned\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.723566 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722862 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-run-netns\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.723566 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722890 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-cni-bin\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.723566 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722916 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-registration-dir\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.723566 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722938 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/833615c3-9d95-448d-a709-4735c575a995-host\") pod \"node-ca-wjpmx\" (UID: \"833615c3-9d95-448d-a709-4735c575a995\") " pod="openshift-image-registry/node-ca-wjpmx" Jul 16 12:07:32.723566 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722958 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-run-netns\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.723566 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722967 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-cni-bin\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.723566 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722962 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-cnibin\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.724232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.722964 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/82627060-8f4c-4020-8e86-506df37a1e55-ovnkube-script-lib\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.724232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723002 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-cnibin\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.724232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723012 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdts7\" (UniqueName: \"kubernetes.io/projected/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-kube-api-access-sdts7\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.724232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723025 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-registration-dir\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.724232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723044 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-systemd\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.724232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723046 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/833615c3-9d95-448d-a709-4735c575a995-host\") pod \"node-ca-wjpmx\" (UID: \"833615c3-9d95-448d-a709-4735c575a995\") " pod="openshift-image-registry/node-ca-wjpmx" Jul 16 12:07:32.724232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723069 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-slash\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.724232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723097 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-systemd\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.724232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723125 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-systemd-units\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.724232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723160 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 12:07:32.724232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723191 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-run-ovn\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.724232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723194 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-systemd-units\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.724232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723221 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.724232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723247 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/82627060-8f4c-4020-8e86-506df37a1e55-env-overrides\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.724232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723224 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-run-ovn\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.725059 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723288 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.725059 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723317 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-run\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.725059 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723349 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jv59\" (UniqueName: \"kubernetes.io/projected/8bdebfdd-8e6f-4154-8d24-9e0c06229293-kube-api-access-4jv59\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.725059 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723378 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-cni-binary-copy\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.725059 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723403 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-run\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.725059 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723404 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret\") pod \"global-pull-secret-syncer-mq859\" (UID: \"a0853c65-0aa1-4d5f-aab4-345f5d30f85f\") " pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:32.725059 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723462 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-system-cni-dir\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.725059 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723519 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-system-cni-dir\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.725059 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723575 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-run-netns\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.725059 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723578 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-system-cni-dir\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.725059 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:32.723597 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:07:32.725059 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723637 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-system-cni-dir\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.725059 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:32.723678 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret podName:a0853c65-0aa1-4d5f-aab4-345f5d30f85f nodeName:}" failed. No retries permitted until 2026-07-16 12:07:33.223655866 +0000 UTC m=+2.987025326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret") pod "global-pull-secret-syncer-mq859" (UID: "a0853c65-0aa1-4d5f-aab4-345f5d30f85f") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:07:32.725059 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723598 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/833615c3-9d95-448d-a709-4735c575a995-serviceca\") pod \"node-ca-wjpmx\" (UID: \"833615c3-9d95-448d-a709-4735c575a995\") " pod="openshift-image-registry/node-ca-wjpmx" Jul 16 12:07:32.725810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723693 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-run-netns\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.725810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723712 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-host\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.725810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723763 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/82627060-8f4c-4020-8e86-506df37a1e55-env-overrides\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.725810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723778 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-var-lib-cni-bin\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.725810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723816 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-var-lib-cni-bin\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.725810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-hostroot\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.725810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723854 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-multus-conf-dir\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.725810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723859 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-host\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.725810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723883 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/374df003-cae9-437a-a50e-2928d6a8b43d-tmp\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.725810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723903 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj8tv\" (UniqueName: \"kubernetes.io/projected/374df003-cae9-437a-a50e-2928d6a8b43d-kube-api-access-nj8tv\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.725810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723911 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-hostroot\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.725810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723917 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-multus-conf-dir\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.725810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.723955 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-run-systemd\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.725810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724001 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a75d700b-84c1-48b2-85bb-dd190d3fd093-cni-binary-copy\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.725810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724028 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-etc-kubernetes\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.726326 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724042 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-cni-binary-copy\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.726326 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724080 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.726326 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724113 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-sysctl-d\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.726326 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724122 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-etc-kubernetes\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.726326 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724136 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/833615c3-9d95-448d-a709-4735c575a995-serviceca\") pod \"node-ca-wjpmx\" (UID: \"833615c3-9d95-448d-a709-4735c575a995\") " pod="openshift-image-registry/node-ca-wjpmx" Jul 16 12:07:32.726326 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724138 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-var-lib-kubelet\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.726326 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724186 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-var-lib-kubelet\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.726326 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724192 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e00ecaab-22f9-4338-ac72-f0eaad408347-hosts-file\") pod \"node-resolver-r7c42\" (UID: \"e00ecaab-22f9-4338-ac72-f0eaad408347\") " pod="openshift-dns/node-resolver-r7c42" Jul 16 12:07:32.726326 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724239 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e00ecaab-22f9-4338-ac72-f0eaad408347-hosts-file\") pod \"node-resolver-r7c42\" (UID: \"e00ecaab-22f9-4338-ac72-f0eaad408347\") " pod="openshift-dns/node-resolver-r7c42" Jul 16 12:07:32.726326 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724230 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-node-log\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.726326 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724259 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-sysctl-d\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.726326 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724029 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-run-systemd\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.726326 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724283 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-log-socket\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.726326 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724289 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-node-log\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.726326 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/82627060-8f4c-4020-8e86-506df37a1e55-ovnkube-config\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.726791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724354 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-cnibin\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.726791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724351 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-log-socket\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.726791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724393 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzp6v\" (UniqueName: \"kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v\") pod \"network-check-target-8lp25\" (UID: \"2d5d88f9-0022-4b17-9d13-8fcbc76ef75d\") " pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:32.726791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724413 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-cnibin\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.726791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724422 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-multus-socket-dir-parent\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.726791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724475 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-multus-socket-dir-parent\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.726791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724600 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-var-lib-cni-multus\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.726791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724626 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.726791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724668 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a75d700b-84c1-48b2-85bb-dd190d3fd093-cni-binary-copy\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.726791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724677 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-var-lib-kubelet\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.726791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724690 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-var-lib-cni-multus\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.726791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724709 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a75d700b-84c1-48b2-85bb-dd190d3fd093-multus-daemon-config\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.726791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724738 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-var-lib-kubelet\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.726791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724765 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-run-multus-certs\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.726791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724810 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpdkr\" (UniqueName: \"kubernetes.io/projected/6f3b3c2d-0128-4354-aa5d-014078c0cd04-kube-api-access-lpdkr\") pod \"iptables-alerter-ss4rt\" (UID: \"6f3b3c2d-0128-4354-aa5d-014078c0cd04\") " pod="openshift-network-operator/iptables-alerter-ss4rt" Jul 16 12:07:32.727230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724830 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-run-multus-certs\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.727230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724847 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6fc5ae7e-11cb-4716-8341-d6613923b14b-agent-certs\") pod \"konnectivity-agent-9p5kr\" (UID: \"6fc5ae7e-11cb-4716-8341-d6613923b14b\") " pod="kube-system/konnectivity-agent-9p5kr" Jul 16 12:07:32.727230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724873 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-socket-dir\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.727230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724900 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-device-dir\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.727230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724944 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.727230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724973 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdqrw\" (UniqueName: \"kubernetes.io/projected/e00ecaab-22f9-4338-ac72-f0eaad408347-kube-api-access-mdqrw\") pod \"node-resolver-r7c42\" (UID: \"e00ecaab-22f9-4338-ac72-f0eaad408347\") " pod="openshift-dns/node-resolver-r7c42" Jul 16 12:07:32.727230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725038 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-device-dir\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.727230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725072 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-socket-dir\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.727230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-var-lib-openvswitch\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.727230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725151 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/82627060-8f4c-4020-8e86-506df37a1e55-ovn-node-metrics-cert\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.727230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725169 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-var-lib-openvswitch\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.727230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725183 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-multus-cni-dir\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.727230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.724946 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/82627060-8f4c-4020-8e86-506df37a1e55-ovnkube-config\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.727230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725209 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-os-release\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.727705 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725249 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-run-k8s-cni-cncf-io\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.727705 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725278 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f3b3c2d-0128-4354-aa5d-014078c0cd04-host-slash\") pod \"iptables-alerter-ss4rt\" (UID: \"6f3b3c2d-0128-4354-aa5d-014078c0cd04\") " pod="openshift-network-operator/iptables-alerter-ss4rt" Jul 16 12:07:32.727705 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725302 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-etc-openvswitch\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.727705 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725311 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a75d700b-84c1-48b2-85bb-dd190d3fd093-multus-daemon-config\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.727705 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725329 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-sysctl-conf\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.727705 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725357 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-etc-selinux\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.727705 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725362 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-host-run-k8s-cni-cncf-io\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.727705 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725373 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f3b3c2d-0128-4354-aa5d-014078c0cd04-host-slash\") pod \"iptables-alerter-ss4rt\" (UID: \"6f3b3c2d-0128-4354-aa5d-014078c0cd04\") " pod="openshift-network-operator/iptables-alerter-ss4rt" Jul 16 12:07:32.727705 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725381 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-sys-fs\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.727705 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725446 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-lib-modules\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.727705 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725453 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82627060-8f4c-4020-8e86-506df37a1e55-etc-openvswitch\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.727705 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725475 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-os-release\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.727705 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725517 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs\") pod \"network-metrics-daemon-rdpjz\" (UID: \"e73a1e88-85a5-4b47-bf8f-372eec90eafd\") " pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:32.727705 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725562 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6f3b3c2d-0128-4354-aa5d-014078c0cd04-iptables-alerter-script\") pod \"iptables-alerter-ss4rt\" (UID: \"6f3b3c2d-0128-4354-aa5d-014078c0cd04\") " pod="openshift-network-operator/iptables-alerter-ss4rt" Jul 16 12:07:32.728245 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725594 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-etc-selinux\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.728245 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725643 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bdebfdd-8e6f-4154-8d24-9e0c06229293-sys-fs\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.728245 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725595 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a75d700b-84c1-48b2-85bb-dd190d3fd093-multus-cni-dir\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.728245 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725650 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.728245 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725695 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-etc-sysctl-conf\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.728245 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:32.725784 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:07:32.728245 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.725798 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/374df003-cae9-437a-a50e-2928d6a8b43d-lib-modules\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.728245 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:32.725939 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs podName:e73a1e88-85a5-4b47-bf8f-372eec90eafd nodeName:}" failed. No retries permitted until 2026-07-16 12:07:33.225920916 +0000 UTC m=+2.989290359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs") pod "network-metrics-daemon-rdpjz" (UID: "e73a1e88-85a5-4b47-bf8f-372eec90eafd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:07:32.728245 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.726171 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6f3b3c2d-0128-4354-aa5d-014078c0cd04-iptables-alerter-script\") pod \"iptables-alerter-ss4rt\" (UID: \"6f3b3c2d-0128-4354-aa5d-014078c0cd04\") " pod="openshift-network-operator/iptables-alerter-ss4rt" Jul 16 12:07:32.728245 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.727726 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/374df003-cae9-437a-a50e-2928d6a8b43d-tmp\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.728245 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.727778 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/374df003-cae9-437a-a50e-2928d6a8b43d-etc-tuned\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.728245 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.728002 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/82627060-8f4c-4020-8e86-506df37a1e55-ovn-node-metrics-cert\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.740222 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.740198 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmbh4\" (UniqueName: \"kubernetes.io/projected/a75d700b-84c1-48b2-85bb-dd190d3fd093-kube-api-access-tmbh4\") pod \"multus-ct6tl\" (UID: \"a75d700b-84c1-48b2-85bb-dd190d3fd093\") " pod="openshift-multus/multus-ct6tl" Jul 16 12:07:32.763297 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.763272 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwdw9\" (UniqueName: \"kubernetes.io/projected/e73a1e88-85a5-4b47-bf8f-372eec90eafd-kube-api-access-jwdw9\") pod \"network-metrics-daemon-rdpjz\" (UID: \"e73a1e88-85a5-4b47-bf8f-372eec90eafd\") " pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:32.764083 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.764058 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b8hv\" (UniqueName: \"kubernetes.io/projected/82627060-8f4c-4020-8e86-506df37a1e55-kube-api-access-8b8hv\") pod \"ovnkube-node-cx284\" (UID: \"82627060-8f4c-4020-8e86-506df37a1e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.764764 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.764733 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdts7\" (UniqueName: \"kubernetes.io/projected/c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47-kube-api-access-sdts7\") pod \"multus-additional-cni-plugins-hcnsq\" (UID: \"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47\") " pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.765155 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:32.765133 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:07:32.765155 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:32.765153 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:07:32.765319 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:32.765166 2578 projected.go:196] Error preparing data for projected volume kube-api-access-qzp6v for pod openshift-network-diagnostics/network-check-target-8lp25: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:07:32.765319 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:32.765237 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v podName:2d5d88f9-0022-4b17-9d13-8fcbc76ef75d nodeName:}" failed. No retries permitted until 2026-07-16 12:07:33.265225051 +0000 UTC m=+3.028594494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qzp6v" (UniqueName: "kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v") pod "network-check-target-8lp25" (UID: "2d5d88f9-0022-4b17-9d13-8fcbc76ef75d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:07:32.766674 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.766651 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj8tv\" (UniqueName: \"kubernetes.io/projected/374df003-cae9-437a-a50e-2928d6a8b43d-kube-api-access-nj8tv\") pod \"tuned-9gd89\" (UID: \"374df003-cae9-437a-a50e-2928d6a8b43d\") " pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.766754 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.766674 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdqrw\" (UniqueName: \"kubernetes.io/projected/e00ecaab-22f9-4338-ac72-f0eaad408347-kube-api-access-mdqrw\") pod \"node-resolver-r7c42\" (UID: \"e00ecaab-22f9-4338-ac72-f0eaad408347\") " pod="openshift-dns/node-resolver-r7c42" Jul 16 12:07:32.767975 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.767958 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jv59\" (UniqueName: \"kubernetes.io/projected/8bdebfdd-8e6f-4154-8d24-9e0c06229293-kube-api-access-4jv59\") pod \"aws-ebs-csi-driver-node-bh5rc\" (UID: \"8bdebfdd-8e6f-4154-8d24-9e0c06229293\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.771413 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.771393 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbn8j\" (UniqueName: \"kubernetes.io/projected/833615c3-9d95-448d-a709-4735c575a995-kube-api-access-zbn8j\") pod \"node-ca-wjpmx\" (UID: \"833615c3-9d95-448d-a709-4735c575a995\") " pod="openshift-image-registry/node-ca-wjpmx" Jul 16 12:07:32.772742 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.772726 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpdkr\" (UniqueName: \"kubernetes.io/projected/6f3b3c2d-0128-4354-aa5d-014078c0cd04-kube-api-access-lpdkr\") pod \"iptables-alerter-ss4rt\" (UID: \"6f3b3c2d-0128-4354-aa5d-014078c0cd04\") " pod="openshift-network-operator/iptables-alerter-ss4rt" Jul 16 12:07:32.799257 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:32.799227 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e3b2dbb5e31a37252e72dba16a64d8f.slice/crio-074bb6b8236a6dd43bcb8ccb0469d753fc7bee5598dcedc4f6d9d954857d105c WatchSource:0}: Error finding container 074bb6b8236a6dd43bcb8ccb0469d753fc7bee5598dcedc4f6d9d954857d105c: Status 404 returned error can't find the container with id 074bb6b8236a6dd43bcb8ccb0469d753fc7bee5598dcedc4f6d9d954857d105c Jul 16 12:07:32.804561 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.804546 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 12:07:32.826737 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.826717 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6fc5ae7e-11cb-4716-8341-d6613923b14b-agent-certs\") pod \"konnectivity-agent-9p5kr\" (UID: \"6fc5ae7e-11cb-4716-8341-d6613923b14b\") " pod="kube-system/konnectivity-agent-9p5kr" Jul 16 12:07:32.826835 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.826772 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6fc5ae7e-11cb-4716-8341-d6613923b14b-konnectivity-ca\") pod \"konnectivity-agent-9p5kr\" (UID: \"6fc5ae7e-11cb-4716-8341-d6613923b14b\") " pod="kube-system/konnectivity-agent-9p5kr" Jul 16 12:07:32.827209 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.827190 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6fc5ae7e-11cb-4716-8341-d6613923b14b-konnectivity-ca\") pod \"konnectivity-agent-9p5kr\" (UID: \"6fc5ae7e-11cb-4716-8341-d6613923b14b\") " pod="kube-system/konnectivity-agent-9p5kr" Jul 16 12:07:32.828921 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.828904 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6fc5ae7e-11cb-4716-8341-d6613923b14b-agent-certs\") pod \"konnectivity-agent-9p5kr\" (UID: \"6fc5ae7e-11cb-4716-8341-d6613923b14b\") " pod="kube-system/konnectivity-agent-9p5kr" Jul 16 12:07:32.872229 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.872147 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 12:02:32 +0000 UTC" deadline="2028-01-29 17:30:51.29550176 +0000 UTC" Jul 16 12:07:32.872229 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.872185 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13493h23m18.42331915s" Jul 16 12:07:32.912766 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.912747 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" Jul 16 12:07:32.916330 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.916316 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9gd89" Jul 16 12:07:32.922229 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:32.922207 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bdebfdd_8e6f_4154_8d24_9e0c06229293.slice/crio-f5b7e4847e5823900ad5e866100c1cafc57d43c2462bf0d2d2c6ae3d6ac38991 WatchSource:0}: Error finding container f5b7e4847e5823900ad5e866100c1cafc57d43c2462bf0d2d2c6ae3d6ac38991: Status 404 returned error can't find the container with id f5b7e4847e5823900ad5e866100c1cafc57d43c2462bf0d2d2c6ae3d6ac38991 Jul 16 12:07:32.925464 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:32.925441 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod374df003_cae9_437a_a50e_2928d6a8b43d.slice/crio-d5f4e8ab40172ff7948c42cfc5512c3166f66c9907a72c059f641192fb642f29 WatchSource:0}: Error finding container d5f4e8ab40172ff7948c42cfc5512c3166f66c9907a72c059f641192fb642f29: Status 404 returned error can't find the container with id d5f4e8ab40172ff7948c42cfc5512c3166f66c9907a72c059f641192fb642f29 Jul 16 12:07:32.937278 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.937259 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r7c42" Jul 16 12:07:32.942059 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.942042 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wjpmx" Jul 16 12:07:32.945390 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:32.945370 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode00ecaab_22f9_4338_ac72_f0eaad408347.slice/crio-3d434384f9bd5e7be83c1740af7db2ca04327400e57faa507527028ac4fabbde WatchSource:0}: Error finding container 3d434384f9bd5e7be83c1740af7db2ca04327400e57faa507527028ac4fabbde: Status 404 returned error can't find the container with id 3d434384f9bd5e7be83c1740af7db2ca04327400e57faa507527028ac4fabbde Jul 16 12:07:32.950521 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:32.950500 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod833615c3_9d95_448d_a709_4735c575a995.slice/crio-4dc487638d5d05de1974b43fd5be1eeb2633289c711af5285f0c82e00bb8bb48 WatchSource:0}: Error finding container 4dc487638d5d05de1974b43fd5be1eeb2633289c711af5285f0c82e00bb8bb48: Status 404 returned error can't find the container with id 4dc487638d5d05de1974b43fd5be1eeb2633289c711af5285f0c82e00bb8bb48 Jul 16 12:07:32.972723 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.972701 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:07:32.974886 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.974864 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hcnsq" Jul 16 12:07:32.982750 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:32.982729 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82627060_8f4c_4020_8e86_506df37a1e55.slice/crio-0306ffbcfdcfc296e2f813d6c6f66c01c1132bfe350253b421a4884933cd0772 WatchSource:0}: Error finding container 0306ffbcfdcfc296e2f813d6c6f66c01c1132bfe350253b421a4884933cd0772: Status 404 returned error can't find the container with id 0306ffbcfdcfc296e2f813d6c6f66c01c1132bfe350253b421a4884933cd0772 Jul 16 12:07:32.985454 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:32.985432 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc04e8ab9_cde7_4a65_8ff1_55ac30fc3f47.slice/crio-c542838182100bde655041b55d6c7c6c67c7a21682ab36e79020b6e859519167 WatchSource:0}: Error finding container c542838182100bde655041b55d6c7c6c67c7a21682ab36e79020b6e859519167: Status 404 returned error can't find the container with id c542838182100bde655041b55d6c7c6c67c7a21682ab36e79020b6e859519167 Jul 16 12:07:32.993448 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:32.993431 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ct6tl" Jul 16 12:07:33.001773 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:33.001756 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda75d700b_84c1_48b2_85bb_dd190d3fd093.slice/crio-6a14a1b592f0bedbd6a0eb4ecf541d517d20ce60b5a263b8639635f456a9c52b WatchSource:0}: Error finding container 6a14a1b592f0bedbd6a0eb4ecf541d517d20ce60b5a263b8639635f456a9c52b: Status 404 returned error can't find the container with id 6a14a1b592f0bedbd6a0eb4ecf541d517d20ce60b5a263b8639635f456a9c52b Jul 16 12:07:33.015856 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.015838 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ss4rt" Jul 16 12:07:33.020759 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.020671 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9p5kr" Jul 16 12:07:33.022826 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:33.022808 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f3b3c2d_0128_4354_aa5d_014078c0cd04.slice/crio-a8e8735e58b6b9cd2c7c20772c242e0efdaf8485ab4304494cd6b9be32119e4b WatchSource:0}: Error finding container a8e8735e58b6b9cd2c7c20772c242e0efdaf8485ab4304494cd6b9be32119e4b: Status 404 returned error can't find the container with id a8e8735e58b6b9cd2c7c20772c242e0efdaf8485ab4304494cd6b9be32119e4b Jul 16 12:07:33.027464 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:33.027444 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fc5ae7e_11cb_4716_8341_d6613923b14b.slice/crio-971e9389a01443bcf45b0d0674f8885b2b4f3cd60b7859b032f0c7791b1f69e5 WatchSource:0}: Error finding container 971e9389a01443bcf45b0d0674f8885b2b4f3cd60b7859b032f0c7791b1f69e5: Status 404 returned error can't find the container with id 971e9389a01443bcf45b0d0674f8885b2b4f3cd60b7859b032f0c7791b1f69e5 Jul 16 12:07:33.040874 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:07:33.040852 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05e2cca6f5dc03b998a60a299686fd5b.slice/crio-aa611fdf40a722468cf73f6d38b69bfa7b1c4100dcb3949420b7e6b560fc2187 WatchSource:0}: Error finding container aa611fdf40a722468cf73f6d38b69bfa7b1c4100dcb3949420b7e6b560fc2187: Status 404 returned error can't find the container with id aa611fdf40a722468cf73f6d38b69bfa7b1c4100dcb3949420b7e6b560fc2187 Jul 16 12:07:33.103991 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.103962 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 12:07:33.229639 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.229561 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret\") pod \"global-pull-secret-syncer-mq859\" (UID: \"a0853c65-0aa1-4d5f-aab4-345f5d30f85f\") " pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:33.229639 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.229630 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs\") pod \"network-metrics-daemon-rdpjz\" (UID: \"e73a1e88-85a5-4b47-bf8f-372eec90eafd\") " pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:33.229815 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:33.229730 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:07:33.229815 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:33.229784 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret podName:a0853c65-0aa1-4d5f-aab4-345f5d30f85f nodeName:}" failed. No retries permitted until 2026-07-16 12:07:34.229770137 +0000 UTC m=+3.993139593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret") pod "global-pull-secret-syncer-mq859" (UID: "a0853c65-0aa1-4d5f-aab4-345f5d30f85f") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:07:33.229815 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:33.229731 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:07:33.229912 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:33.229822 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs podName:e73a1e88-85a5-4b47-bf8f-372eec90eafd nodeName:}" failed. No retries permitted until 2026-07-16 12:07:34.229812671 +0000 UTC m=+3.993182114 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs") pod "network-metrics-daemon-rdpjz" (UID: "e73a1e88-85a5-4b47-bf8f-372eec90eafd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:07:33.330293 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.330260 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzp6v\" (UniqueName: \"kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v\") pod \"network-check-target-8lp25\" (UID: \"2d5d88f9-0022-4b17-9d13-8fcbc76ef75d\") " pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:33.330505 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:33.330484 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:07:33.330582 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:33.330513 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:07:33.330582 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:33.330527 2578 projected.go:196] Error preparing data for projected volume kube-api-access-qzp6v for pod openshift-network-diagnostics/network-check-target-8lp25: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:07:33.330696 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:33.330599 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v podName:2d5d88f9-0022-4b17-9d13-8fcbc76ef75d nodeName:}" failed. No retries permitted until 2026-07-16 12:07:34.330580287 +0000 UTC m=+4.093949732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzp6v" (UniqueName: "kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v") pod "network-check-target-8lp25" (UID: "2d5d88f9-0022-4b17-9d13-8fcbc76ef75d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:07:33.489874 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.489734 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9gd89" event={"ID":"374df003-cae9-437a-a50e-2928d6a8b43d","Type":"ContainerStarted","Data":"d5f4e8ab40172ff7948c42cfc5512c3166f66c9907a72c059f641192fb642f29"} Jul 16 12:07:33.500527 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.500438 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" event={"ID":"8bdebfdd-8e6f-4154-8d24-9e0c06229293","Type":"ContainerStarted","Data":"f5b7e4847e5823900ad5e866100c1cafc57d43c2462bf0d2d2c6ae3d6ac38991"} Jul 16 12:07:33.510193 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.510112 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9p5kr" event={"ID":"6fc5ae7e-11cb-4716-8341-d6613923b14b","Type":"ContainerStarted","Data":"971e9389a01443bcf45b0d0674f8885b2b4f3cd60b7859b032f0c7791b1f69e5"} Jul 16 12:07:33.520351 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.520317 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wjpmx" event={"ID":"833615c3-9d95-448d-a709-4735c575a995","Type":"ContainerStarted","Data":"4dc487638d5d05de1974b43fd5be1eeb2633289c711af5285f0c82e00bb8bb48"} Jul 16 12:07:33.525098 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.525020 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-160.ec2.internal" event={"ID":"4e3b2dbb5e31a37252e72dba16a64d8f","Type":"ContainerStarted","Data":"074bb6b8236a6dd43bcb8ccb0469d753fc7bee5598dcedc4f6d9d954857d105c"} Jul 16 12:07:33.528719 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.528634 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal" event={"ID":"05e2cca6f5dc03b998a60a299686fd5b","Type":"ContainerStarted","Data":"aa611fdf40a722468cf73f6d38b69bfa7b1c4100dcb3949420b7e6b560fc2187"} Jul 16 12:07:33.543565 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.543532 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ss4rt" event={"ID":"6f3b3c2d-0128-4354-aa5d-014078c0cd04","Type":"ContainerStarted","Data":"a8e8735e58b6b9cd2c7c20772c242e0efdaf8485ab4304494cd6b9be32119e4b"} Jul 16 12:07:33.546014 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.545980 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ct6tl" event={"ID":"a75d700b-84c1-48b2-85bb-dd190d3fd093","Type":"ContainerStarted","Data":"6a14a1b592f0bedbd6a0eb4ecf541d517d20ce60b5a263b8639635f456a9c52b"} Jul 16 12:07:33.550524 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.550472 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcnsq" event={"ID":"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47","Type":"ContainerStarted","Data":"c542838182100bde655041b55d6c7c6c67c7a21682ab36e79020b6e859519167"} Jul 16 12:07:33.558102 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.557954 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" event={"ID":"82627060-8f4c-4020-8e86-506df37a1e55","Type":"ContainerStarted","Data":"0306ffbcfdcfc296e2f813d6c6f66c01c1132bfe350253b421a4884933cd0772"} Jul 16 12:07:33.565331 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.565305 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r7c42" event={"ID":"e00ecaab-22f9-4338-ac72-f0eaad408347","Type":"ContainerStarted","Data":"3d434384f9bd5e7be83c1740af7db2ca04327400e57faa507527028ac4fabbde"} Jul 16 12:07:33.872714 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.872572 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 12:02:32 +0000 UTC" deadline="2027-12-19 07:25:06.150461896 +0000 UTC" Jul 16 12:07:33.872714 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:33.872631 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12499h17m32.277834838s" Jul 16 12:07:34.238122 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:34.238032 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret\") pod \"global-pull-secret-syncer-mq859\" (UID: \"a0853c65-0aa1-4d5f-aab4-345f5d30f85f\") " pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:34.238296 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:34.238122 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs\") pod \"network-metrics-daemon-rdpjz\" (UID: \"e73a1e88-85a5-4b47-bf8f-372eec90eafd\") " pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:34.238296 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:34.238249 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:07:34.238435 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:34.238346 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs podName:e73a1e88-85a5-4b47-bf8f-372eec90eafd nodeName:}" failed. No retries permitted until 2026-07-16 12:07:36.238296183 +0000 UTC m=+6.001665629 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs") pod "network-metrics-daemon-rdpjz" (UID: "e73a1e88-85a5-4b47-bf8f-372eec90eafd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:07:34.238916 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:34.238895 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:07:34.239028 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:34.238958 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret podName:a0853c65-0aa1-4d5f-aab4-345f5d30f85f nodeName:}" failed. No retries permitted until 2026-07-16 12:07:36.238941418 +0000 UTC m=+6.002310880 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret") pod "global-pull-secret-syncer-mq859" (UID: "a0853c65-0aa1-4d5f-aab4-345f5d30f85f") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:07:34.339147 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:34.339110 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzp6v\" (UniqueName: \"kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v\") pod \"network-check-target-8lp25\" (UID: \"2d5d88f9-0022-4b17-9d13-8fcbc76ef75d\") " pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:34.339334 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:34.339311 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:07:34.339399 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:34.339338 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:07:34.339399 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:34.339351 2578 projected.go:196] Error preparing data for projected volume kube-api-access-qzp6v for pod openshift-network-diagnostics/network-check-target-8lp25: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:07:34.339534 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:34.339421 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v podName:2d5d88f9-0022-4b17-9d13-8fcbc76ef75d nodeName:}" failed. No retries permitted until 2026-07-16 12:07:36.339400693 +0000 UTC m=+6.102770140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzp6v" (UniqueName: "kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v") pod "network-check-target-8lp25" (UID: "2d5d88f9-0022-4b17-9d13-8fcbc76ef75d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:07:34.473055 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:34.473016 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:34.473296 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:34.473187 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:07:34.473943 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:34.473622 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:34.473943 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:34.473724 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:07:34.473943 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:34.473795 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:34.473943 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:34.473871 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:07:36.257763 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:36.256979 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs\") pod \"network-metrics-daemon-rdpjz\" (UID: \"e73a1e88-85a5-4b47-bf8f-372eec90eafd\") " pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:36.257763 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:36.257053 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret\") pod \"global-pull-secret-syncer-mq859\" (UID: \"a0853c65-0aa1-4d5f-aab4-345f5d30f85f\") " pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:36.257763 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:36.257189 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:07:36.257763 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:36.257253 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret podName:a0853c65-0aa1-4d5f-aab4-345f5d30f85f nodeName:}" failed. No retries permitted until 2026-07-16 12:07:40.257233311 +0000 UTC m=+10.020602755 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret") pod "global-pull-secret-syncer-mq859" (UID: "a0853c65-0aa1-4d5f-aab4-345f5d30f85f") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:07:36.257763 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:36.257674 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:07:36.257763 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:36.257728 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs podName:e73a1e88-85a5-4b47-bf8f-372eec90eafd nodeName:}" failed. No retries permitted until 2026-07-16 12:07:40.25771192 +0000 UTC m=+10.021081366 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs") pod "network-metrics-daemon-rdpjz" (UID: "e73a1e88-85a5-4b47-bf8f-372eec90eafd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:07:36.358314 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:36.358244 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzp6v\" (UniqueName: \"kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v\") pod \"network-check-target-8lp25\" (UID: \"2d5d88f9-0022-4b17-9d13-8fcbc76ef75d\") " pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:36.358552 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:36.358429 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:07:36.358552 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:36.358452 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:07:36.358552 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:36.358486 2578 projected.go:196] Error preparing data for projected volume kube-api-access-qzp6v for pod openshift-network-diagnostics/network-check-target-8lp25: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:07:36.358552 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:36.358550 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v podName:2d5d88f9-0022-4b17-9d13-8fcbc76ef75d nodeName:}" failed. No retries permitted until 2026-07-16 12:07:40.358528799 +0000 UTC m=+10.121898245 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzp6v" (UniqueName: "kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v") pod "network-check-target-8lp25" (UID: "2d5d88f9-0022-4b17-9d13-8fcbc76ef75d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:07:36.474180 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:36.473341 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:36.474180 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:36.473516 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:07:36.474180 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:36.473909 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:36.474180 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:36.474004 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:07:36.474180 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:36.474053 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:36.474180 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:36.474130 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:07:38.474185 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:38.473915 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:38.474185 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:38.474083 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:07:38.474754 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:38.474428 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:38.474754 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:38.474540 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:38.474754 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:38.474564 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:07:38.474754 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:38.474698 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:07:40.293910 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:40.293864 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret\") pod \"global-pull-secret-syncer-mq859\" (UID: \"a0853c65-0aa1-4d5f-aab4-345f5d30f85f\") " pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:40.294421 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:40.293979 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs\") pod \"network-metrics-daemon-rdpjz\" (UID: \"e73a1e88-85a5-4b47-bf8f-372eec90eafd\") " pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:40.294421 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:40.294101 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:07:40.294421 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:40.294179 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs podName:e73a1e88-85a5-4b47-bf8f-372eec90eafd nodeName:}" failed. No retries permitted until 2026-07-16 12:07:48.294156052 +0000 UTC m=+18.057525498 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs") pod "network-metrics-daemon-rdpjz" (UID: "e73a1e88-85a5-4b47-bf8f-372eec90eafd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:07:40.294421 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:40.294101 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:07:40.294421 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:40.294397 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret podName:a0853c65-0aa1-4d5f-aab4-345f5d30f85f nodeName:}" failed. No retries permitted until 2026-07-16 12:07:48.294378377 +0000 UTC m=+18.057747834 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret") pod "global-pull-secret-syncer-mq859" (UID: "a0853c65-0aa1-4d5f-aab4-345f5d30f85f") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:07:40.395114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:40.395025 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzp6v\" (UniqueName: \"kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v\") pod \"network-check-target-8lp25\" (UID: \"2d5d88f9-0022-4b17-9d13-8fcbc76ef75d\") " pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:40.395300 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:40.395214 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:07:40.395300 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:40.395237 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:07:40.395300 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:40.395247 2578 projected.go:196] Error preparing data for projected volume kube-api-access-qzp6v for pod openshift-network-diagnostics/network-check-target-8lp25: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:07:40.395427 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:40.395308 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v podName:2d5d88f9-0022-4b17-9d13-8fcbc76ef75d nodeName:}" failed. No retries permitted until 2026-07-16 12:07:48.395288938 +0000 UTC m=+18.158658382 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzp6v" (UniqueName: "kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v") pod "network-check-target-8lp25" (UID: "2d5d88f9-0022-4b17-9d13-8fcbc76ef75d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:07:40.473642 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:40.473602 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:40.473836 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:40.473602 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:40.473836 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:40.473617 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:40.473836 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:40.473761 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:07:40.474009 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:40.473854 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:07:40.474009 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:40.473968 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:07:42.473863 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:42.473815 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:42.473863 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:42.473842 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:42.474476 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:42.473815 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:42.474476 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:42.473996 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:07:42.474476 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:42.474119 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:07:42.474476 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:42.474204 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:07:44.473816 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:44.473784 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:44.474257 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:44.473783 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:44.474257 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:44.473908 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:44.474257 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:44.473903 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:07:44.474257 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:44.474000 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:07:44.474257 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:44.474114 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:07:46.473244 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:46.473202 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:46.473750 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:46.473202 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:46.473750 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:46.473327 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:07:46.473750 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:46.473429 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:07:46.473750 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:46.473217 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:46.473750 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:46.473524 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:07:48.359141 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:48.359100 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs\") pod \"network-metrics-daemon-rdpjz\" (UID: \"e73a1e88-85a5-4b47-bf8f-372eec90eafd\") " pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:48.359604 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:48.359165 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret\") pod \"global-pull-secret-syncer-mq859\" (UID: \"a0853c65-0aa1-4d5f-aab4-345f5d30f85f\") " pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:48.359604 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:48.359264 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:07:48.359604 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:48.359330 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret podName:a0853c65-0aa1-4d5f-aab4-345f5d30f85f nodeName:}" failed. No retries permitted until 2026-07-16 12:08:04.35931584 +0000 UTC m=+34.122685281 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret") pod "global-pull-secret-syncer-mq859" (UID: "a0853c65-0aa1-4d5f-aab4-345f5d30f85f") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:07:48.359604 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:48.359264 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:07:48.359604 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:48.359423 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs podName:e73a1e88-85a5-4b47-bf8f-372eec90eafd nodeName:}" failed. No retries permitted until 2026-07-16 12:08:04.359395173 +0000 UTC m=+34.122764618 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs") pod "network-metrics-daemon-rdpjz" (UID: "e73a1e88-85a5-4b47-bf8f-372eec90eafd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:07:48.460258 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:48.460220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzp6v\" (UniqueName: \"kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v\") pod \"network-check-target-8lp25\" (UID: \"2d5d88f9-0022-4b17-9d13-8fcbc76ef75d\") " pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:48.460466 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:48.460370 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:07:48.460466 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:48.460392 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:07:48.460466 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:48.460404 2578 projected.go:196] Error preparing data for projected volume kube-api-access-qzp6v for pod openshift-network-diagnostics/network-check-target-8lp25: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:07:48.460630 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:48.460485 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v podName:2d5d88f9-0022-4b17-9d13-8fcbc76ef75d nodeName:}" failed. No retries permitted until 2026-07-16 12:08:04.460466849 +0000 UTC m=+34.223836304 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzp6v" (UniqueName: "kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v") pod "network-check-target-8lp25" (UID: "2d5d88f9-0022-4b17-9d13-8fcbc76ef75d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:07:48.473954 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:48.473927 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:48.474090 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:48.473955 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:48.474090 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:48.473963 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:48.474090 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:48.474048 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:07:48.474239 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:48.474128 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:07:48.474239 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:48.474199 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:07:50.473407 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:50.473370 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:50.473407 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:50.473397 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:50.473994 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:50.473513 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:50.473994 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:50.473522 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:07:50.473994 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:50.473615 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:07:50.473994 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:50.473707 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:07:52.473772 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:52.473732 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:52.474228 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:52.473780 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:52.474228 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:52.473746 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:52.474228 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:52.473882 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:07:52.474228 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:52.473954 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:07:52.474228 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:52.474029 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:07:54.473706 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:54.473673 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:54.474056 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:54.473717 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:54.474056 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:54.473674 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:54.474056 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:54.473822 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:07:54.474056 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:54.473942 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:07:54.474056 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:54.474007 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:07:54.622452 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:54.622371 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ct6tl" event={"ID":"a75d700b-84c1-48b2-85bb-dd190d3fd093","Type":"ContainerStarted","Data":"e38f04b1e9b5726b7747cc4357a6654d12d9b31bb272b6c6f5f038674c13fb56"} Jul 16 12:07:54.625624 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:54.625334 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:07:54.625723 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:54.625698 2578 generic.go:356] "Generic (PLEG): container finished" podID="82627060-8f4c-4020-8e86-506df37a1e55" containerID="b37a475db25fa04078557d0c10d5b4a9790adcdda1838284394298315d443b16" exitCode=1 Jul 16 12:07:54.625808 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:54.625784 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" event={"ID":"82627060-8f4c-4020-8e86-506df37a1e55","Type":"ContainerStarted","Data":"ff9ae7bacae028e226e95773ffe9d7743b1fa55f91580563c5759de9373f96c5"} Jul 16 12:07:54.625876 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:54.625861 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" event={"ID":"82627060-8f4c-4020-8e86-506df37a1e55","Type":"ContainerStarted","Data":"98d2c612ca15dce861cf7fd60416a06de20159fd1e19979f400adda33670a00f"} Jul 16 12:07:54.625954 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:54.625920 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" event={"ID":"82627060-8f4c-4020-8e86-506df37a1e55","Type":"ContainerDied","Data":"b37a475db25fa04078557d0c10d5b4a9790adcdda1838284394298315d443b16"} Jul 16 12:07:54.625988 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:54.625976 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" event={"ID":"82627060-8f4c-4020-8e86-506df37a1e55","Type":"ContainerStarted","Data":"af0151b38230f90272b8b9a6a08f830225ab6c7ba38ac5af17e4ddbeb47fcb02"} Jul 16 12:07:54.627353 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:54.627331 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9gd89" event={"ID":"374df003-cae9-437a-a50e-2928d6a8b43d","Type":"ContainerStarted","Data":"fb2d184af63f94801a2d3815b5b5d397c44c7e78b36fe7631d92e50cfa72b3e7"} Jul 16 12:07:54.628544 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:54.628521 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-160.ec2.internal" event={"ID":"4e3b2dbb5e31a37252e72dba16a64d8f","Type":"ContainerStarted","Data":"ae28e2b7c0fece635ce921a494b8472f8d144861ab36dd13c1cac73fb8ef603f"} Jul 16 12:07:54.649223 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:54.649039 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ct6tl" podStartSLOduration=1.517828687 podStartE2EDuration="22.64902555s" totalImagesPullingTime="21.131196863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:07:54.648978843 +0000 UTC m=+24.412348306" watchObservedRunningTime="2026-07-16 12:07:54.64902555 +0000 UTC m=+24.412395013" Jul 16 12:07:54.739581 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:54.739521 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9gd89" podStartSLOduration=1.53135657 podStartE2EDuration="22.739495138s" totalImagesPullingTime="21.208138568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:07:54.739171814 +0000 UTC m=+24.502541278" watchObservedRunningTime="2026-07-16 12:07:54.739495138 +0000 UTC m=+24.502864601" Jul 16 12:07:54.740056 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:54.739966 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-160.ec2.internal" podStartSLOduration=22.73995764 podStartE2EDuration="22.73995764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:07:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:07:54.69789125 +0000 UTC m=+24.461260713" watchObservedRunningTime="2026-07-16 12:07:54.73995764 +0000 UTC m=+24.503327097" Jul 16 12:07:55.631901 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:55.631862 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r7c42" event={"ID":"e00ecaab-22f9-4338-ac72-f0eaad408347","Type":"ContainerStarted","Data":"ead84867db6c8e7980e29ce147fdda171e129d0098d270b9238c15e4d83fff88"} Jul 16 12:07:55.633241 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:55.633218 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" event={"ID":"8bdebfdd-8e6f-4154-8d24-9e0c06229293","Type":"ContainerStarted","Data":"e0aa0e12d3d90b6de3f50baf2b6e5844a95d3e4d72c5ee34642e3fdb7600ce5c"} Jul 16 12:07:55.634507 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:55.634487 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9p5kr" event={"ID":"6fc5ae7e-11cb-4716-8341-d6613923b14b","Type":"ContainerStarted","Data":"3ca237d86e65bedcefafe4b5f20e2e12404e8b0d76e96f1ade998c2aa2a0c784"} Jul 16 12:07:55.635775 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:55.635752 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wjpmx" event={"ID":"833615c3-9d95-448d-a709-4735c575a995","Type":"ContainerStarted","Data":"79844efb857bd7edacb18768e37beb18b81c50198e68105d1e68e8e047b42687"} Jul 16 12:07:55.637111 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:55.637089 2578 generic.go:356] "Generic (PLEG): container finished" podID="05e2cca6f5dc03b998a60a299686fd5b" containerID="1cbc67e0500977bd78116499053f61f2da9da8a35d5d3640dbfbc6e995d5ac18" exitCode=0 Jul 16 12:07:55.637205 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:55.637151 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal" event={"ID":"05e2cca6f5dc03b998a60a299686fd5b","Type":"ContainerDied","Data":"1cbc67e0500977bd78116499053f61f2da9da8a35d5d3640dbfbc6e995d5ac18"} Jul 16 12:07:55.638510 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:55.638490 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ss4rt" event={"ID":"6f3b3c2d-0128-4354-aa5d-014078c0cd04","Type":"ContainerStarted","Data":"9969b2e6451acb060a1f076c936063ffc9cdb45965e24ae47602a6367c4c7f11"} Jul 16 12:07:55.639770 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:55.639751 2578 generic.go:356] "Generic (PLEG): container finished" podID="c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47" containerID="359f04640f716a831922066e4e02193787f0bfcd106d31d7bdbfe171c4fac6ac" exitCode=0 Jul 16 12:07:55.639854 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:55.639814 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcnsq" event={"ID":"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47","Type":"ContainerDied","Data":"359f04640f716a831922066e4e02193787f0bfcd106d31d7bdbfe171c4fac6ac"} Jul 16 12:07:55.642539 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:55.642524 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:07:55.642889 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:55.642860 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" event={"ID":"82627060-8f4c-4020-8e86-506df37a1e55","Type":"ContainerStarted","Data":"be62c4bda4c8ec0e239442743a0277ce77c12db4408e7c332c6c19100b9d705d"} Jul 16 12:07:55.642987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:55.642929 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" event={"ID":"82627060-8f4c-4020-8e86-506df37a1e55","Type":"ContainerStarted","Data":"69ac54898153248cca625b0663f65c8e59e0f0dd876a8fd639d448c7e2176882"} Jul 16 12:07:55.692719 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:55.692673 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-r7c42" podStartSLOduration=2.50431245 podStartE2EDuration="23.692655545s" totalImagesPullingTime="21.188343095s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:07:55.661203943 +0000 UTC m=+25.424573406" watchObservedRunningTime="2026-07-16 12:07:55.692655545 +0000 UTC m=+25.456025008" Jul 16 12:07:55.723485 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:55.723388 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ss4rt" podStartSLOduration=2.648900549 podStartE2EDuration="23.723367655s" totalImagesPullingTime="21.074467106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:07:55.692789635 +0000 UTC m=+25.456159076" watchObservedRunningTime="2026-07-16 12:07:55.723367655 +0000 UTC m=+25.486737118" Jul 16 12:07:55.828424 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:55.828359 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wjpmx" podStartSLOduration=2.68191959 podStartE2EDuration="23.828343286s" totalImagesPullingTime="21.146423696s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:07:55.723039092 +0000 UTC m=+25.486408569" watchObservedRunningTime="2026-07-16 12:07:55.828343286 +0000 UTC m=+25.591712750" Jul 16 12:07:56.046073 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.045994 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9p5kr" podStartSLOduration=2.9795869440000002 podStartE2EDuration="24.045974745s" totalImagesPullingTime="21.066387801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:07:55.98188707 +0000 UTC m=+25.745256535" watchObservedRunningTime="2026-07-16 12:07:56.045974745 +0000 UTC m=+25.809344207" Jul 16 12:07:56.059980 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.059952 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 12:07:56.281532 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.281455 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:07:56.319031 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.318992 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9p5kr" Jul 16 12:07:56.319559 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.319540 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9p5kr" Jul 16 12:07:56.351029 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.350932 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T12:07:56.059975424Z","UUID":"a85ad8c3-c681-494c-8a38-b71eb2276daf","Handler":null,"Name":"","Endpoint":""} Jul 16 12:07:56.352849 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.352825 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 12:07:56.352963 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.352872 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 12:07:56.394908 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.394868 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:07:56.473989 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.473960 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:56.474206 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.473960 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:56.474206 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:56.474097 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:07:56.474206 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.473960 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:56.474206 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:56.474174 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:07:56.474442 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:56.474220 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:07:56.648343 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.648258 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal" event={"ID":"05e2cca6f5dc03b998a60a299686fd5b","Type":"ContainerStarted","Data":"8ed3a8edb05ee10913a743853aaa102832aba5ddd6e91be343fcc8590be1c31e"} Jul 16 12:07:56.650308 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.650278 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" event={"ID":"8bdebfdd-8e6f-4154-8d24-9e0c06229293","Type":"ContainerStarted","Data":"3601f7c809016c432a152a66bcdec8a3c8c2c354a6ebc4bde8fca24cf8d8400d"} Jul 16 12:07:56.650631 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.650603 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9p5kr" Jul 16 12:07:56.651124 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.651105 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9p5kr" Jul 16 12:07:56.676608 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.676547 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-160.ec2.internal" podStartSLOduration=24.676526542 podStartE2EDuration="24.676526542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:07:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:07:56.675758415 +0000 UTC m=+26.439127878" watchObservedRunningTime="2026-07-16 12:07:56.676526542 +0000 UTC m=+26.439896054" Jul 16 12:07:56.816076 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:56.816039 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:07:57.656067 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:57.656033 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:07:57.656557 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:57.656407 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" event={"ID":"82627060-8f4c-4020-8e86-506df37a1e55","Type":"ContainerStarted","Data":"328753d8ddc6adc9ad03493de0ba89752bacae573596d6e60cea4dae99638006"} Jul 16 12:07:57.658844 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:57.658817 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" event={"ID":"8bdebfdd-8e6f-4154-8d24-9e0c06229293","Type":"ContainerStarted","Data":"06eb8bd0d0a317203d97f5d042cab1f94ba3e8317c441940ebde33aa7106e823"} Jul 16 12:07:57.693135 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:57.693072 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bh5rc" podStartSLOduration=1.793843983 podStartE2EDuration="25.693049962s" totalImagesPullingTime="23.899205979s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:07:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:07:57.691774777 +0000 UTC m=+27.455144241" watchObservedRunningTime="2026-07-16 12:07:57.693049962 +0000 UTC m=+27.456419424" Jul 16 12:07:58.473704 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:58.473519 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:07:58.473866 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:58.473554 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:07:58.473866 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:07:58.473584 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:07:58.473866 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:58.473810 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:07:58.474019 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:58.473897 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:07:58.474019 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:07:58.473998 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:08:00.473937 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:00.473753 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:08:00.474625 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:00.473754 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:08:00.474625 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:00.474011 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:08:00.474625 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:00.473760 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:08:00.474625 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:00.474080 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:08:00.474625 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:00.474145 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:08:00.670932 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:00.670895 2578 generic.go:356] "Generic (PLEG): container finished" podID="c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47" containerID="3294382b825d1c53798f3eb5acc73934207feeda44e5b9c82da131fd89052963" exitCode=0 Jul 16 12:08:00.671116 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:00.670989 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcnsq" event={"ID":"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47","Type":"ContainerDied","Data":"3294382b825d1c53798f3eb5acc73934207feeda44e5b9c82da131fd89052963"} Jul 16 12:08:00.674284 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:00.674268 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:08:00.674609 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:00.674589 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" event={"ID":"82627060-8f4c-4020-8e86-506df37a1e55","Type":"ContainerStarted","Data":"ffa97924804b2e19b404ed7dcf8b6aa5d034e59b9a247811f2cb6ac8e89ade38"} Jul 16 12:08:00.674850 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:00.674830 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:08:00.674919 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:00.674884 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:08:00.675003 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:00.674989 2578 scope.go:117] "RemoveContainer" containerID="b37a475db25fa04078557d0c10d5b4a9790adcdda1838284394298315d443b16" Jul 16 12:08:00.690882 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:00.690859 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:08:01.680931 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:01.680907 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:08:01.681401 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:01.681284 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" event={"ID":"82627060-8f4c-4020-8e86-506df37a1e55","Type":"ContainerStarted","Data":"a72b6f9cbce414c1f3b2121d1ae21f42e3cb522139ef170e75856dcf9a233e53"} Jul 16 12:08:01.681629 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:01.681610 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:08:01.697976 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:01.697952 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:08:01.739362 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:01.739279 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" podStartSLOduration=8.523231492 podStartE2EDuration="29.739261203s" totalImagesPullingTime="21.216029711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:08:01.739186819 +0000 UTC m=+31.502556283" watchObservedRunningTime="2026-07-16 12:08:01.739261203 +0000 UTC m=+31.502630665" Jul 16 12:08:02.473427 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:02.473386 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:08:02.473602 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:02.473433 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:08:02.473602 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:02.473400 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:08:02.473602 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:02.473543 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:08:02.473710 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:02.473660 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:08:02.473753 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:02.473732 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:08:02.492028 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:02.492002 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8lp25"] Jul 16 12:08:02.499914 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:02.499843 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rdpjz"] Jul 16 12:08:02.500474 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:02.500431 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mq859"] Jul 16 12:08:02.685367 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:02.685325 2578 generic.go:356] "Generic (PLEG): container finished" podID="c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47" containerID="f0ea510e4e12ffa1d16f89ecfb23c1a025bc16757feb97e1b3467fe36d0f21d5" exitCode=0 Jul 16 12:08:02.685851 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:02.685387 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcnsq" event={"ID":"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47","Type":"ContainerDied","Data":"f0ea510e4e12ffa1d16f89ecfb23c1a025bc16757feb97e1b3467fe36d0f21d5"} Jul 16 12:08:02.685851 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:02.685403 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:08:02.685851 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:02.685500 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:08:02.685851 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:02.685610 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:08:02.685851 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:02.685658 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:08:02.685851 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:02.685692 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:08:02.685851 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:02.685753 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:08:03.996273 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:03.996232 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:08:04.379573 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:04.379471 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret\") pod \"global-pull-secret-syncer-mq859\" (UID: \"a0853c65-0aa1-4d5f-aab4-345f5d30f85f\") " pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:08:04.379745 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:04.379561 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs\") pod \"network-metrics-daemon-rdpjz\" (UID: \"e73a1e88-85a5-4b47-bf8f-372eec90eafd\") " pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:08:04.379745 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:04.379627 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:08:04.379745 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:04.379666 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:08:04.379745 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:04.379699 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret podName:a0853c65-0aa1-4d5f-aab4-345f5d30f85f nodeName:}" failed. No retries permitted until 2026-07-16 12:08:36.379679658 +0000 UTC m=+66.143049113 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret") pod "global-pull-secret-syncer-mq859" (UID: "a0853c65-0aa1-4d5f-aab4-345f5d30f85f") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:08:04.379745 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:04.379716 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs podName:e73a1e88-85a5-4b47-bf8f-372eec90eafd nodeName:}" failed. No retries permitted until 2026-07-16 12:08:36.379708461 +0000 UTC m=+66.143077902 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs") pod "network-metrics-daemon-rdpjz" (UID: "e73a1e88-85a5-4b47-bf8f-372eec90eafd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:08:04.473957 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:04.473916 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:08:04.473957 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:04.473954 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:08:04.474160 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:04.474017 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:08:04.474160 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:04.474032 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:08:04.474160 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:04.474085 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:08:04.474269 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:04.474166 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:08:04.480130 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:04.480109 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzp6v\" (UniqueName: \"kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v\") pod \"network-check-target-8lp25\" (UID: \"2d5d88f9-0022-4b17-9d13-8fcbc76ef75d\") " pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:08:04.480249 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:04.480235 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:08:04.480291 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:04.480252 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:08:04.480291 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:04.480261 2578 projected.go:196] Error preparing data for projected volume kube-api-access-qzp6v for pod openshift-network-diagnostics/network-check-target-8lp25: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:08:04.480356 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:04.480306 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v podName:2d5d88f9-0022-4b17-9d13-8fcbc76ef75d nodeName:}" failed. No retries permitted until 2026-07-16 12:08:36.480290169 +0000 UTC m=+66.243659623 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzp6v" (UniqueName: "kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v") pod "network-check-target-8lp25" (UID: "2d5d88f9-0022-4b17-9d13-8fcbc76ef75d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:08:05.197765 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:05.197737 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:08:05.693334 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:05.693307 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcnsq" event={"ID":"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47","Type":"ContainerStarted","Data":"b577e06dd59b16b54f01290df7509546fd1b6a28651e4ad56f10efe882407d35"} Jul 16 12:08:06.473368 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:06.473316 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:08:06.473815 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:06.473319 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:08:06.473815 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:06.473474 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lp25" podUID="2d5d88f9-0022-4b17-9d13-8fcbc76ef75d" Jul 16 12:08:06.473815 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:06.473514 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mq859" podUID="a0853c65-0aa1-4d5f-aab4-345f5d30f85f" Jul 16 12:08:06.473815 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:06.473327 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:08:06.473815 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:06.473661 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:08:06.700905 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:06.700873 2578 generic.go:356] "Generic (PLEG): container finished" podID="c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47" containerID="b577e06dd59b16b54f01290df7509546fd1b6a28651e4ad56f10efe882407d35" exitCode=0 Jul 16 12:08:06.701086 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:06.700935 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcnsq" event={"ID":"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47","Type":"ContainerDied","Data":"b577e06dd59b16b54f01290df7509546fd1b6a28651e4ad56f10efe882407d35"} Jul 16 12:08:07.107110 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.107084 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-160.ec2.internal" event="NodeReady" Jul 16 12:08:07.107271 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.107213 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 12:08:07.201345 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.201317 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qblpk"] Jul 16 12:08:07.203080 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.203057 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:07.207094 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.207067 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 12:08:07.207252 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.207151 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 12:08:07.207327 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.207292 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v8966\"" Jul 16 12:08:07.208545 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.208527 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-knfq6"] Jul 16 12:08:07.210381 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.210362 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-knfq6" Jul 16 12:08:07.216074 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.216051 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 12:08:07.216175 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.216051 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 12:08:07.217966 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.217947 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 12:08:07.219963 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.219949 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pwb6c\"" Jul 16 12:08:07.241024 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.240998 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-knfq6"] Jul 16 12:08:07.295772 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.295743 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qblpk"] Jul 16 12:08:07.301192 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.301171 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:07.301310 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.301205 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77wjq\" (UniqueName: \"kubernetes.io/projected/8174e182-bed0-49f1-bdae-aeaaa73fd83c-kube-api-access-77wjq\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:07.301310 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.301239 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dn5j\" (UniqueName: \"kubernetes.io/projected/10ea27e0-362a-4adb-a93c-e3db335e4610-kube-api-access-8dn5j\") pod \"ingress-canary-knfq6\" (UID: \"10ea27e0-362a-4adb-a93c-e3db335e4610\") " pod="openshift-ingress-canary/ingress-canary-knfq6" Jul 16 12:08:07.301401 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.301322 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8174e182-bed0-49f1-bdae-aeaaa73fd83c-config-volume\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:07.301401 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.301348 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10ea27e0-362a-4adb-a93c-e3db335e4610-cert\") pod \"ingress-canary-knfq6\" (UID: \"10ea27e0-362a-4adb-a93c-e3db335e4610\") " pod="openshift-ingress-canary/ingress-canary-knfq6" Jul 16 12:08:07.301401 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.301366 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8174e182-bed0-49f1-bdae-aeaaa73fd83c-tmp-dir\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:07.402135 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.402065 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77wjq\" (UniqueName: \"kubernetes.io/projected/8174e182-bed0-49f1-bdae-aeaaa73fd83c-kube-api-access-77wjq\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:07.402135 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.402118 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dn5j\" (UniqueName: \"kubernetes.io/projected/10ea27e0-362a-4adb-a93c-e3db335e4610-kube-api-access-8dn5j\") pod \"ingress-canary-knfq6\" (UID: \"10ea27e0-362a-4adb-a93c-e3db335e4610\") " pod="openshift-ingress-canary/ingress-canary-knfq6" Jul 16 12:08:07.402307 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.402289 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8174e182-bed0-49f1-bdae-aeaaa73fd83c-config-volume\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:07.402347 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.402320 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10ea27e0-362a-4adb-a93c-e3db335e4610-cert\") pod \"ingress-canary-knfq6\" (UID: \"10ea27e0-362a-4adb-a93c-e3db335e4610\") " pod="openshift-ingress-canary/ingress-canary-knfq6" Jul 16 12:08:07.402347 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.402342 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8174e182-bed0-49f1-bdae-aeaaa73fd83c-tmp-dir\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:07.402408 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.402394 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:07.402513 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:07.402502 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 12:08:07.402558 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:07.402549 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls podName:8174e182-bed0-49f1-bdae-aeaaa73fd83c nodeName:}" failed. No retries permitted until 2026-07-16 12:08:07.902536039 +0000 UTC m=+37.665905480 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls") pod "dns-default-qblpk" (UID: "8174e182-bed0-49f1-bdae-aeaaa73fd83c") : secret "dns-default-metrics-tls" not found Jul 16 12:08:07.402711 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.402696 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8174e182-bed0-49f1-bdae-aeaaa73fd83c-tmp-dir\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:07.402892 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.402873 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8174e182-bed0-49f1-bdae-aeaaa73fd83c-config-volume\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:07.406553 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.406531 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10ea27e0-362a-4adb-a93c-e3db335e4610-cert\") pod \"ingress-canary-knfq6\" (UID: \"10ea27e0-362a-4adb-a93c-e3db335e4610\") " pod="openshift-ingress-canary/ingress-canary-knfq6" Jul 16 12:08:07.418934 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.418911 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dn5j\" (UniqueName: \"kubernetes.io/projected/10ea27e0-362a-4adb-a93c-e3db335e4610-kube-api-access-8dn5j\") pod \"ingress-canary-knfq6\" (UID: \"10ea27e0-362a-4adb-a93c-e3db335e4610\") " pod="openshift-ingress-canary/ingress-canary-knfq6" Jul 16 12:08:07.419920 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.419902 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77wjq\" (UniqueName: \"kubernetes.io/projected/8174e182-bed0-49f1-bdae-aeaaa73fd83c-kube-api-access-77wjq\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:07.497471 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.497440 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:08:07.518596 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.518570 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-knfq6" Jul 16 12:08:07.612106 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.612072 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:08:07.689494 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.689385 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-knfq6"] Jul 16 12:08:07.697123 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:08:07.697097 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ea27e0_362a_4adb_a93c_e3db335e4610.slice/crio-9546220cd6080cf57f3b8973efe0381362cc25708d4402fdc9b04acbf3702664 WatchSource:0}: Error finding container 9546220cd6080cf57f3b8973efe0381362cc25708d4402fdc9b04acbf3702664: Status 404 returned error can't find the container with id 9546220cd6080cf57f3b8973efe0381362cc25708d4402fdc9b04acbf3702664 Jul 16 12:08:07.703865 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.703838 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-knfq6" event={"ID":"10ea27e0-362a-4adb-a93c-e3db335e4610","Type":"ContainerStarted","Data":"9546220cd6080cf57f3b8973efe0381362cc25708d4402fdc9b04acbf3702664"} Jul 16 12:08:07.908596 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:07.908553 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:07.908788 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:07.908721 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 12:08:07.908850 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:07.908801 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls podName:8174e182-bed0-49f1-bdae-aeaaa73fd83c nodeName:}" failed. No retries permitted until 2026-07-16 12:08:08.908780086 +0000 UTC m=+38.672149529 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls") pod "dns-default-qblpk" (UID: "8174e182-bed0-49f1-bdae-aeaaa73fd83c") : secret "dns-default-metrics-tls" not found Jul 16 12:08:08.474005 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:08.473763 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:08:08.474195 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:08.473812 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:08:08.474195 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:08.473812 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:08:08.478698 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:08.478675 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 12:08:08.479363 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:08.479340 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hz6m9\"" Jul 16 12:08:08.479363 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:08.479357 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 12:08:08.479567 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:08.479371 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 12:08:08.479567 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:08.479361 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 12:08:08.479697 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:08.479683 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w29rm\"" Jul 16 12:08:08.917066 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:08.916962 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:08.917797 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:08.917115 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 12:08:08.917797 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:08.917193 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls podName:8174e182-bed0-49f1-bdae-aeaaa73fd83c nodeName:}" failed. No retries permitted until 2026-07-16 12:08:10.917173431 +0000 UTC m=+40.680542878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls") pod "dns-default-qblpk" (UID: "8174e182-bed0-49f1-bdae-aeaaa73fd83c") : secret "dns-default-metrics-tls" not found Jul 16 12:08:10.712057 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:10.711954 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-knfq6" event={"ID":"10ea27e0-362a-4adb-a93c-e3db335e4610","Type":"ContainerStarted","Data":"d65e79e19a5087690f7e49f6034c7bc002d0092743d7dda211066ec22c5dd9d0"} Jul 16 12:08:10.735565 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:10.735506 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-knfq6" podStartSLOduration=1.154911587 podStartE2EDuration="3.735480476s" totalImagesPullingTime="2.580568889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:08:10.735161236 +0000 UTC m=+40.498530701" watchObservedRunningTime="2026-07-16 12:08:10.735480476 +0000 UTC m=+40.498849937" Jul 16 12:08:10.933576 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:10.933544 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:10.933819 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:10.933684 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 12:08:10.933819 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:10.933744 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls podName:8174e182-bed0-49f1-bdae-aeaaa73fd83c nodeName:}" failed. No retries permitted until 2026-07-16 12:08:14.933728536 +0000 UTC m=+44.697097989 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls") pod "dns-default-qblpk" (UID: "8174e182-bed0-49f1-bdae-aeaaa73fd83c") : secret "dns-default-metrics-tls" not found Jul 16 12:08:11.516300 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:11.516270 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:08:13.721761 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:13.721682 2578 generic.go:356] "Generic (PLEG): container finished" podID="c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47" containerID="c3b933030b018ee4c592db552104eccb6dcd3e5056ce092f6150c5b047d473e2" exitCode=0 Jul 16 12:08:13.721761 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:13.721738 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcnsq" event={"ID":"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47","Type":"ContainerDied","Data":"c3b933030b018ee4c592db552104eccb6dcd3e5056ce092f6150c5b047d473e2"} Jul 16 12:08:14.726957 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:14.726920 2578 generic.go:356] "Generic (PLEG): container finished" podID="c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47" containerID="b8e80e189c46b7194556909455a6fe58d64d36c67a255ee25dd6f50e71c9b523" exitCode=0 Jul 16 12:08:14.727337 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:14.726988 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcnsq" event={"ID":"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47","Type":"ContainerDied","Data":"b8e80e189c46b7194556909455a6fe58d64d36c67a255ee25dd6f50e71c9b523"} Jul 16 12:08:14.967206 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:14.967169 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:14.967370 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:14.967317 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 12:08:14.967441 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:14.967380 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls podName:8174e182-bed0-49f1-bdae-aeaaa73fd83c nodeName:}" failed. No retries permitted until 2026-07-16 12:08:22.967365071 +0000 UTC m=+52.730734512 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls") pod "dns-default-qblpk" (UID: "8174e182-bed0-49f1-bdae-aeaaa73fd83c") : secret "dns-default-metrics-tls" not found Jul 16 12:08:15.732557 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:15.732525 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcnsq" event={"ID":"c04e8ab9-cde7-4a65-8ff1-55ac30fc3f47","Type":"ContainerStarted","Data":"df67ef110896bf77e05b5ab0385c4edb2d0e3ab74b5ccd4bd835f3ddc4287095"} Jul 16 12:08:15.773472 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:15.773403 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hcnsq" podStartSLOduration=8.001902526 podStartE2EDuration="43.773386735s" totalImagesPullingTime="35.771484209s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:07:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:08:15.773374566 +0000 UTC m=+45.536744028" watchObservedRunningTime="2026-07-16 12:08:15.773386735 +0000 UTC m=+45.536756197" Jul 16 12:08:16.059850 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:16.059824 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:08:16.144981 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:16.144941 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:08:16.257055 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:16.257026 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:08:23.023676 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:23.023636 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:23.024238 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:23.023814 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 12:08:23.024238 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:23.023900 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls podName:8174e182-bed0-49f1-bdae-aeaaa73fd83c nodeName:}" failed. No retries permitted until 2026-07-16 12:08:39.023877879 +0000 UTC m=+68.787247320 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls") pod "dns-default-qblpk" (UID: "8174e182-bed0-49f1-bdae-aeaaa73fd83c") : secret "dns-default-metrics-tls" not found Jul 16 12:08:33.697794 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:33.697767 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cx284" Jul 16 12:08:33.855699 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:33.855663 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:08:33.956928 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:33.956822 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:08:36.408168 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:36.408115 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret\") pod \"global-pull-secret-syncer-mq859\" (UID: \"a0853c65-0aa1-4d5f-aab4-345f5d30f85f\") " pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:08:36.408677 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:36.408216 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs\") pod \"network-metrics-daemon-rdpjz\" (UID: \"e73a1e88-85a5-4b47-bf8f-372eec90eafd\") " pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:08:36.411937 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:36.411919 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 12:08:36.411990 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:36.411947 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 12:08:36.418643 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:36.418623 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 12:08:36.418735 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:36.418706 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs podName:e73a1e88-85a5-4b47-bf8f-372eec90eafd nodeName:}" failed. No retries permitted until 2026-07-16 12:09:40.418683605 +0000 UTC m=+130.182053059 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs") pod "network-metrics-daemon-rdpjz" (UID: "e73a1e88-85a5-4b47-bf8f-372eec90eafd") : secret "metrics-daemon-secret" not found Jul 16 12:08:36.421768 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:36.421748 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0853c65-0aa1-4d5f-aab4-345f5d30f85f-original-pull-secret\") pod \"global-pull-secret-syncer-mq859\" (UID: \"a0853c65-0aa1-4d5f-aab4-345f5d30f85f\") " pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:08:36.508814 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:36.508767 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzp6v\" (UniqueName: \"kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v\") pod \"network-check-target-8lp25\" (UID: \"2d5d88f9-0022-4b17-9d13-8fcbc76ef75d\") " pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:08:36.511354 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:36.511335 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 12:08:36.521399 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:36.521381 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 12:08:36.531875 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:36.531856 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzp6v\" (UniqueName: \"kubernetes.io/projected/2d5d88f9-0022-4b17-9d13-8fcbc76ef75d-kube-api-access-qzp6v\") pod \"network-check-target-8lp25\" (UID: \"2d5d88f9-0022-4b17-9d13-8fcbc76ef75d\") " pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:08:36.686556 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:36.686480 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mq859" Jul 16 12:08:36.700039 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:36.700015 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hz6m9\"" Jul 16 12:08:36.707820 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:36.707799 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:08:36.827644 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:36.827616 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mq859"] Jul 16 12:08:36.831429 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:08:36.831387 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0853c65_0aa1_4d5f_aab4_345f5d30f85f.slice/crio-ba202427062fc4494fa74b7b82aa1a97b4e21ea69ebdb4da16c799b99976bb74 WatchSource:0}: Error finding container ba202427062fc4494fa74b7b82aa1a97b4e21ea69ebdb4da16c799b99976bb74: Status 404 returned error can't find the container with id ba202427062fc4494fa74b7b82aa1a97b4e21ea69ebdb4da16c799b99976bb74 Jul 16 12:08:36.843069 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:36.843045 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8lp25"] Jul 16 12:08:36.846124 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:08:36.846106 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d5d88f9_0022_4b17_9d13_8fcbc76ef75d.slice/crio-bec7e8cec3125b873620943d1817378feaf699d071c23041338dfd695509da1d WatchSource:0}: Error finding container bec7e8cec3125b873620943d1817378feaf699d071c23041338dfd695509da1d: Status 404 returned error can't find the container with id bec7e8cec3125b873620943d1817378feaf699d071c23041338dfd695509da1d Jul 16 12:08:37.786567 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:37.786527 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8lp25" event={"ID":"2d5d88f9-0022-4b17-9d13-8fcbc76ef75d","Type":"ContainerStarted","Data":"bec7e8cec3125b873620943d1817378feaf699d071c23041338dfd695509da1d"} Jul 16 12:08:37.787874 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:37.787843 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mq859" event={"ID":"a0853c65-0aa1-4d5f-aab4-345f5d30f85f","Type":"ContainerStarted","Data":"ba202427062fc4494fa74b7b82aa1a97b4e21ea69ebdb4da16c799b99976bb74"} Jul 16 12:08:39.028682 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:39.028642 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:08:39.029151 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:39.028778 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 12:08:39.029151 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:08:39.028884 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls podName:8174e182-bed0-49f1-bdae-aeaaa73fd83c nodeName:}" failed. No retries permitted until 2026-07-16 12:09:11.028865285 +0000 UTC m=+100.792234742 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls") pod "dns-default-qblpk" (UID: "8174e182-bed0-49f1-bdae-aeaaa73fd83c") : secret "dns-default-metrics-tls" not found Jul 16 12:08:39.794935 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:39.794894 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8lp25" event={"ID":"2d5d88f9-0022-4b17-9d13-8fcbc76ef75d","Type":"ContainerStarted","Data":"b07d69341b1d3d2f84a4a5e3d610342ab50f4560b4b5b3bd2d31371d16d86a48"} Jul 16 12:08:39.795120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:39.795042 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:08:39.814819 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:39.814688 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8lp25" podStartSLOduration=65.10910373 podStartE2EDuration="1m7.814668118s" totalImagesPullingTime="2.705564388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:08:39.814221276 +0000 UTC m=+69.577590740" watchObservedRunningTime="2026-07-16 12:08:39.814668118 +0000 UTC m=+69.578037584" Jul 16 12:08:42.804296 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:42.804258 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mq859" event={"ID":"a0853c65-0aa1-4d5f-aab4-345f5d30f85f","Type":"ContainerStarted","Data":"84a13be688bba3ea915a22c02dcef0d7f278514acf9409bf01e0119872c52e46"} Jul 16 12:08:42.825835 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:08:42.825788 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mq859" podStartSLOduration=65.224473917 podStartE2EDuration="1m10.825772851s" totalImagesPullingTime="5.601298934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:08:42.825513996 +0000 UTC m=+72.588883462" watchObservedRunningTime="2026-07-16 12:08:42.825772851 +0000 UTC m=+72.589142346" Jul 16 12:09:10.800269 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:09:10.800166 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8lp25" Jul 16 12:09:11.047619 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:09:11.047568 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:09:11.047804 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:09:11.047691 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 12:09:11.047804 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:09:11.047751 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls podName:8174e182-bed0-49f1-bdae-aeaaa73fd83c nodeName:}" failed. No retries permitted until 2026-07-16 12:10:15.047737486 +0000 UTC m=+164.811106927 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls") pod "dns-default-qblpk" (UID: "8174e182-bed0-49f1-bdae-aeaaa73fd83c") : secret "dns-default-metrics-tls" not found Jul 16 12:09:40.453273 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:09:40.453220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs\") pod \"network-metrics-daemon-rdpjz\" (UID: \"e73a1e88-85a5-4b47-bf8f-372eec90eafd\") " pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:09:40.453713 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:09:40.453360 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 12:09:40.453713 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:09:40.453431 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs podName:e73a1e88-85a5-4b47-bf8f-372eec90eafd nodeName:}" failed. No retries permitted until 2026-07-16 12:11:42.453400021 +0000 UTC m=+252.216769465 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs") pod "network-metrics-daemon-rdpjz" (UID: "e73a1e88-85a5-4b47-bf8f-372eec90eafd") : secret "metrics-daemon-secret" not found Jul 16 12:10:06.474042 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:06.473987 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:10.212628 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:10.212588 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qblpk" podUID="8174e182-bed0-49f1-bdae-aeaaa73fd83c" Jul 16 12:10:11.009243 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:11.009214 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qblpk" Jul 16 12:10:11.354893 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:11.354865 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:11.493216 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:11.493173 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rdpjz" podUID="e73a1e88-85a5-4b47-bf8f-372eec90eafd" Jul 16 12:10:15.090238 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:15.090180 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:10:15.090733 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:15.090332 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 12:10:15.090733 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:15.090408 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls podName:8174e182-bed0-49f1-bdae-aeaaa73fd83c nodeName:}" failed. No retries permitted until 2026-07-16 12:12:17.090392738 +0000 UTC m=+286.853762182 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls") pod "dns-default-qblpk" (UID: "8174e182-bed0-49f1-bdae-aeaaa73fd83c") : secret "dns-default-metrics-tls" not found Jul 16 12:10:24.473697 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:24.473589 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:10:38.667217 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.667178 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c99fx"] Jul 16 12:10:38.669128 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.669107 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-68956567b4-ccm7k"] Jul 16 12:10:38.669424 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.669389 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c99fx" Jul 16 12:10:38.670911 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.670893 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:38.671878 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.671859 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-d7cch\"" Jul 16 12:10:38.673143 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.673126 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 16 12:10:38.674052 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.674037 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 12:10:38.674110 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.674073 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 12:10:38.675851 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.675830 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 16 12:10:38.675951 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.675937 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-sc2vk\"" Jul 16 12:10:38.675994 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.675974 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 16 12:10:38.677406 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.677377 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 16 12:10:38.677528 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.677404 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 16 12:10:38.679241 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.679218 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 16 12:10:38.698587 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.698561 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c99fx"] Jul 16 12:10:38.712901 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.712858 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68956567b4-ccm7k"] Jul 16 12:10:38.723950 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.723922 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-r97s6"] Jul 16 12:10:38.725736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.725721 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r97s6" Jul 16 12:10:38.728819 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.728801 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-44c9g\"" Jul 16 12:10:38.736339 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.736314 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-72p8q"] Jul 16 12:10:38.738361 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.738342 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd"] Jul 16 12:10:38.738489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.738470 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:10:38.739936 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.739915 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-277mm"] Jul 16 12:10:38.740032 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.740018 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd" Jul 16 12:10:38.741714 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.741697 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.742045 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.742028 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 16 12:10:38.742197 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.742181 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-lhxxx\"" Jul 16 12:10:38.742246 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.742233 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 16 12:10:38.744087 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.744069 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 16 12:10:38.746280 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.746101 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 16 12:10:38.746376 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.746357 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 16 12:10:38.746622 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.746605 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 16 12:10:38.746719 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.746657 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 16 12:10:38.746946 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.746926 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 12:10:38.747319 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.747162 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lpwdh\"" Jul 16 12:10:38.747319 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.747176 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-wlcpc\"" Jul 16 12:10:38.747319 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.747199 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 16 12:10:38.747319 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.747295 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-r97s6"] Jul 16 12:10:38.749211 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.749191 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 16 12:10:38.749946 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.749592 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 16 12:10:38.750093 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.750075 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 12:10:38.754146 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.754127 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 16 12:10:38.756066 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.756041 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 16 12:10:38.764798 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.764771 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-277mm"] Jul 16 12:10:38.765501 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.765482 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd"] Jul 16 12:10:38.766069 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.766048 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-72p8q"] Jul 16 12:10:38.857047 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857012 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f73a8a43-dd26-4e85-865c-a1aba38315c0-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.857047 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857044 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmbpq\" (UniqueName: \"kubernetes.io/projected/c72b21b6-dfa9-49e6-b546-b0ac30e2216b-kube-api-access-rmbpq\") pod \"console-operator-7866497494-72p8q\" (UID: \"c72b21b6-dfa9-49e6-b546-b0ac30e2216b\") " pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:10:38.857263 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857066 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdb05941-6cba-40f2-8150-7465f6b82a78-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-rcrbd\" (UID: \"fdb05941-6cba-40f2-8150-7465f6b82a78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd" Jul 16 12:10:38.857263 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857082 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f73a8a43-dd26-4e85-865c-a1aba38315c0-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.857263 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857144 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c72b21b6-dfa9-49e6-b546-b0ac30e2216b-serving-cert\") pod \"console-operator-7866497494-72p8q\" (UID: \"c72b21b6-dfa9-49e6-b546-b0ac30e2216b\") " pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:10:38.857263 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857225 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdb05941-6cba-40f2-8150-7465f6b82a78-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-rcrbd\" (UID: \"fdb05941-6cba-40f2-8150-7465f6b82a78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd" Jul 16 12:10:38.857263 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857246 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f73a8a43-dd26-4e85-865c-a1aba38315c0-tmp\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.857479 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857277 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-stats-auth\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:38.857479 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857295 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c72b21b6-dfa9-49e6-b546-b0ac30e2216b-config\") pod \"console-operator-7866497494-72p8q\" (UID: \"c72b21b6-dfa9-49e6-b546-b0ac30e2216b\") " pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:10:38.857479 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857316 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqjkh\" (UniqueName: \"kubernetes.io/projected/1c7ed060-2647-40fb-b395-6654ac05cd42-kube-api-access-tqjkh\") pod \"volume-data-source-validator-78495597f6-c99fx\" (UID: \"1c7ed060-2647-40fb-b395-6654ac05cd42\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c99fx" Jul 16 12:10:38.857479 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857332 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f73a8a43-dd26-4e85-865c-a1aba38315c0-serving-cert\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.857479 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857382 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9rst\" (UniqueName: \"kubernetes.io/projected/8376ac44-2259-464a-beed-20afd82ac00a-kube-api-access-w9rst\") pod \"network-check-source-869dcfcdcd-r97s6\" (UID: \"8376ac44-2259-464a-beed-20afd82ac00a\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r97s6" Jul 16 12:10:38.857479 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857401 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm6l6\" (UniqueName: \"kubernetes.io/projected/f73a8a43-dd26-4e85-865c-a1aba38315c0-kube-api-access-zm6l6\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.857479 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857442 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:38.857826 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857497 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f73a8a43-dd26-4e85-865c-a1aba38315c0-snapshots\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.857826 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857515 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-default-certificate\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:38.857826 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857538 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5m6v\" (UniqueName: \"kubernetes.io/projected/fdb05941-6cba-40f2-8150-7465f6b82a78-kube-api-access-h5m6v\") pod \"kube-storage-version-migrator-operator-799cb7f958-rcrbd\" (UID: \"fdb05941-6cba-40f2-8150-7465f6b82a78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd" Jul 16 12:10:38.857826 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857553 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:38.857826 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857569 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7gpt\" (UniqueName: \"kubernetes.io/projected/060e6722-ce48-468c-aa89-cc07918db901-kube-api-access-j7gpt\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:38.857826 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.857584 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c72b21b6-dfa9-49e6-b546-b0ac30e2216b-trusted-ca\") pod \"console-operator-7866497494-72p8q\" (UID: \"c72b21b6-dfa9-49e6-b546-b0ac30e2216b\") " pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:10:38.958789 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.958672 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f73a8a43-dd26-4e85-865c-a1aba38315c0-snapshots\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.958789 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.958726 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-default-certificate\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:38.958789 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.958748 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5m6v\" (UniqueName: \"kubernetes.io/projected/fdb05941-6cba-40f2-8150-7465f6b82a78-kube-api-access-h5m6v\") pod \"kube-storage-version-migrator-operator-799cb7f958-rcrbd\" (UID: \"fdb05941-6cba-40f2-8150-7465f6b82a78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd" Jul 16 12:10:38.959051 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.958862 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:38.959051 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.958892 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7gpt\" (UniqueName: \"kubernetes.io/projected/060e6722-ce48-468c-aa89-cc07918db901-kube-api-access-j7gpt\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:38.959051 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.958911 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c72b21b6-dfa9-49e6-b546-b0ac30e2216b-trusted-ca\") pod \"console-operator-7866497494-72p8q\" (UID: \"c72b21b6-dfa9-49e6-b546-b0ac30e2216b\") " pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:10:38.959051 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:38.958994 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 12:10:38.959248 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959048 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f73a8a43-dd26-4e85-865c-a1aba38315c0-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.959248 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:38.959066 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs podName:060e6722-ce48-468c-aa89-cc07918db901 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:39.459041783 +0000 UTC m=+189.222411242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs") pod "router-default-68956567b4-ccm7k" (UID: "060e6722-ce48-468c-aa89-cc07918db901") : secret "router-metrics-certs-default" not found Jul 16 12:10:38.959248 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959102 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmbpq\" (UniqueName: \"kubernetes.io/projected/c72b21b6-dfa9-49e6-b546-b0ac30e2216b-kube-api-access-rmbpq\") pod \"console-operator-7866497494-72p8q\" (UID: \"c72b21b6-dfa9-49e6-b546-b0ac30e2216b\") " pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:10:38.959248 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959134 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdb05941-6cba-40f2-8150-7465f6b82a78-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-rcrbd\" (UID: \"fdb05941-6cba-40f2-8150-7465f6b82a78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd" Jul 16 12:10:38.959248 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959158 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f73a8a43-dd26-4e85-865c-a1aba38315c0-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.959248 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959180 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c72b21b6-dfa9-49e6-b546-b0ac30e2216b-serving-cert\") pod \"console-operator-7866497494-72p8q\" (UID: \"c72b21b6-dfa9-49e6-b546-b0ac30e2216b\") " pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:10:38.959248 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959238 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdb05941-6cba-40f2-8150-7465f6b82a78-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-rcrbd\" (UID: \"fdb05941-6cba-40f2-8150-7465f6b82a78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd" Jul 16 12:10:38.959601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959262 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f73a8a43-dd26-4e85-865c-a1aba38315c0-tmp\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.959601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959291 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-stats-auth\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:38.959601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959317 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c72b21b6-dfa9-49e6-b546-b0ac30e2216b-config\") pod \"console-operator-7866497494-72p8q\" (UID: \"c72b21b6-dfa9-49e6-b546-b0ac30e2216b\") " pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:10:38.959601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959345 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqjkh\" (UniqueName: \"kubernetes.io/projected/1c7ed060-2647-40fb-b395-6654ac05cd42-kube-api-access-tqjkh\") pod \"volume-data-source-validator-78495597f6-c99fx\" (UID: \"1c7ed060-2647-40fb-b395-6654ac05cd42\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c99fx" Jul 16 12:10:38.959601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f73a8a43-dd26-4e85-865c-a1aba38315c0-serving-cert\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.959601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959457 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9rst\" (UniqueName: \"kubernetes.io/projected/8376ac44-2259-464a-beed-20afd82ac00a-kube-api-access-w9rst\") pod \"network-check-source-869dcfcdcd-r97s6\" (UID: \"8376ac44-2259-464a-beed-20afd82ac00a\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r97s6" Jul 16 12:10:38.959601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959487 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm6l6\" (UniqueName: \"kubernetes.io/projected/f73a8a43-dd26-4e85-865c-a1aba38315c0-kube-api-access-zm6l6\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.959601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959514 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:38.959601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959545 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f73a8a43-dd26-4e85-865c-a1aba38315c0-snapshots\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.960066 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:38.959664 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle podName:060e6722-ce48-468c-aa89-cc07918db901 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:39.459646463 +0000 UTC m=+189.223015906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle") pod "router-default-68956567b4-ccm7k" (UID: "060e6722-ce48-468c-aa89-cc07918db901") : configmap references non-existent config key: service-ca.crt Jul 16 12:10:38.960066 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959806 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f73a8a43-dd26-4e85-865c-a1aba38315c0-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.960066 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959827 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdb05941-6cba-40f2-8150-7465f6b82a78-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-rcrbd\" (UID: \"fdb05941-6cba-40f2-8150-7465f6b82a78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd" Jul 16 12:10:38.960066 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.959887 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c72b21b6-dfa9-49e6-b546-b0ac30e2216b-trusted-ca\") pod \"console-operator-7866497494-72p8q\" (UID: \"c72b21b6-dfa9-49e6-b546-b0ac30e2216b\") " pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:10:38.960275 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.960066 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f73a8a43-dd26-4e85-865c-a1aba38315c0-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.960328 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.960279 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f73a8a43-dd26-4e85-865c-a1aba38315c0-tmp\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.960674 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.960649 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c72b21b6-dfa9-49e6-b546-b0ac30e2216b-config\") pod \"console-operator-7866497494-72p8q\" (UID: \"c72b21b6-dfa9-49e6-b546-b0ac30e2216b\") " pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:10:38.961987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.961955 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdb05941-6cba-40f2-8150-7465f6b82a78-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-rcrbd\" (UID: \"fdb05941-6cba-40f2-8150-7465f6b82a78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd" Jul 16 12:10:38.962165 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.962145 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c72b21b6-dfa9-49e6-b546-b0ac30e2216b-serving-cert\") pod \"console-operator-7866497494-72p8q\" (UID: \"c72b21b6-dfa9-49e6-b546-b0ac30e2216b\") " pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:10:38.962220 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.962152 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-default-certificate\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:38.962289 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.962265 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f73a8a43-dd26-4e85-865c-a1aba38315c0-serving-cert\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:38.962496 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:38.962482 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-stats-auth\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:39.004097 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.004069 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:39.018610 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.018580 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5m6v\" (UniqueName: \"kubernetes.io/projected/fdb05941-6cba-40f2-8150-7465f6b82a78-kube-api-access-h5m6v\") pod \"kube-storage-version-migrator-operator-799cb7f958-rcrbd\" (UID: \"fdb05941-6cba-40f2-8150-7465f6b82a78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd" Jul 16 12:10:39.018741 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.018644 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmbpq\" (UniqueName: \"kubernetes.io/projected/c72b21b6-dfa9-49e6-b546-b0ac30e2216b-kube-api-access-rmbpq\") pod \"console-operator-7866497494-72p8q\" (UID: \"c72b21b6-dfa9-49e6-b546-b0ac30e2216b\") " pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:10:39.018741 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.018645 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqjkh\" (UniqueName: \"kubernetes.io/projected/1c7ed060-2647-40fb-b395-6654ac05cd42-kube-api-access-tqjkh\") pod \"volume-data-source-validator-78495597f6-c99fx\" (UID: \"1c7ed060-2647-40fb-b395-6654ac05cd42\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c99fx" Jul 16 12:10:39.018741 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.018657 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm6l6\" (UniqueName: \"kubernetes.io/projected/f73a8a43-dd26-4e85-865c-a1aba38315c0-kube-api-access-zm6l6\") pod \"insights-operator-68d9dd94cf-277mm\" (UID: \"f73a8a43-dd26-4e85-865c-a1aba38315c0\") " pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:39.028233 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.028209 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7gpt\" (UniqueName: \"kubernetes.io/projected/060e6722-ce48-468c-aa89-cc07918db901-kube-api-access-j7gpt\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:39.038198 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.038169 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9rst\" (UniqueName: \"kubernetes.io/projected/8376ac44-2259-464a-beed-20afd82ac00a-kube-api-access-w9rst\") pod \"network-check-source-869dcfcdcd-r97s6\" (UID: \"8376ac44-2259-464a-beed-20afd82ac00a\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r97s6" Jul 16 12:10:39.051126 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.051099 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:10:39.057658 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.057638 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-277mm" Jul 16 12:10:39.061348 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.061329 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd" Jul 16 12:10:39.104658 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.104622 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:39.194308 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.194270 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-72p8q"] Jul 16 12:10:39.197687 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:10:39.197663 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc72b21b6_dfa9_49e6_b546_b0ac30e2216b.slice/crio-6f352d4675f36a13df6a4c6b5a5c5ca63fce0be9ce3b1fedce79a84a870ee4f6 WatchSource:0}: Error finding container 6f352d4675f36a13df6a4c6b5a5c5ca63fce0be9ce3b1fedce79a84a870ee4f6: Status 404 returned error can't find the container with id 6f352d4675f36a13df6a4c6b5a5c5ca63fce0be9ce3b1fedce79a84a870ee4f6 Jul 16 12:10:39.283674 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.283648 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c99fx" Jul 16 12:10:39.334504 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.334467 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r97s6" Jul 16 12:10:39.418191 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.418160 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c99fx"] Jul 16 12:10:39.420451 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.420315 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd"] Jul 16 12:10:39.421475 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.421453 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-277mm"] Jul 16 12:10:39.422165 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:10:39.422138 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c7ed060_2647_40fb_b395_6654ac05cd42.slice/crio-6930f39975555dcc6d4c30ed263fe2b0e227ffd3fcb545b7b6388d4368dee2d4 WatchSource:0}: Error finding container 6930f39975555dcc6d4c30ed263fe2b0e227ffd3fcb545b7b6388d4368dee2d4: Status 404 returned error can't find the container with id 6930f39975555dcc6d4c30ed263fe2b0e227ffd3fcb545b7b6388d4368dee2d4 Jul 16 12:10:39.423336 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:10:39.423294 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdb05941_6cba_40f2_8150_7465f6b82a78.slice/crio-7c4b9e412b5a2205d6aa5e9b1ab37bba30b02031598ea2aa689354c8f1a1ea35 WatchSource:0}: Error finding container 7c4b9e412b5a2205d6aa5e9b1ab37bba30b02031598ea2aa689354c8f1a1ea35: Status 404 returned error can't find the container with id 7c4b9e412b5a2205d6aa5e9b1ab37bba30b02031598ea2aa689354c8f1a1ea35 Jul 16 12:10:39.424267 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:10:39.424235 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf73a8a43_dd26_4e85_865c_a1aba38315c0.slice/crio-76625a816a4a4a90a6747db9a080c79b7b2c4a9c0122c1f5a5ab787f8f57834c WatchSource:0}: Error finding container 76625a816a4a4a90a6747db9a080c79b7b2c4a9c0122c1f5a5ab787f8f57834c: Status 404 returned error can't find the container with id 76625a816a4a4a90a6747db9a080c79b7b2c4a9c0122c1f5a5ab787f8f57834c Jul 16 12:10:39.464002 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.463974 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:39.464182 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.464019 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:39.464182 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:39.464126 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 12:10:39.464182 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:39.464150 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle podName:060e6722-ce48-468c-aa89-cc07918db901 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:40.464129789 +0000 UTC m=+190.227499249 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle") pod "router-default-68956567b4-ccm7k" (UID: "060e6722-ce48-468c-aa89-cc07918db901") : configmap references non-existent config key: service-ca.crt Jul 16 12:10:39.464182 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:39.464175 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs podName:060e6722-ce48-468c-aa89-cc07918db901 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:40.46416749 +0000 UTC m=+190.227536930 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs") pod "router-default-68956567b4-ccm7k" (UID: "060e6722-ce48-468c-aa89-cc07918db901") : secret "router-metrics-certs-default" not found Jul 16 12:10:39.471644 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:39.471620 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-r97s6"] Jul 16 12:10:39.474807 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:10:39.474780 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8376ac44_2259_464a_beed_20afd82ac00a.slice/crio-026e73e43878936955a079d9347a11010d96014821d57fa45abb083ba96543ec WatchSource:0}: Error finding container 026e73e43878936955a079d9347a11010d96014821d57fa45abb083ba96543ec: Status 404 returned error can't find the container with id 026e73e43878936955a079d9347a11010d96014821d57fa45abb083ba96543ec Jul 16 12:10:40.078456 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:40.078405 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r97s6" event={"ID":"8376ac44-2259-464a-beed-20afd82ac00a","Type":"ContainerStarted","Data":"e3533ec83a754d6ff0e97708741fdd96cf745033a1ed4874516f156592bc249a"} Jul 16 12:10:40.078928 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:40.078476 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r97s6" event={"ID":"8376ac44-2259-464a-beed-20afd82ac00a","Type":"ContainerStarted","Data":"026e73e43878936955a079d9347a11010d96014821d57fa45abb083ba96543ec"} Jul 16 12:10:40.079913 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:40.079879 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c99fx" event={"ID":"1c7ed060-2647-40fb-b395-6654ac05cd42","Type":"ContainerStarted","Data":"6930f39975555dcc6d4c30ed263fe2b0e227ffd3fcb545b7b6388d4368dee2d4"} Jul 16 12:10:40.081068 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:40.081041 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd" event={"ID":"fdb05941-6cba-40f2-8150-7465f6b82a78","Type":"ContainerStarted","Data":"7c4b9e412b5a2205d6aa5e9b1ab37bba30b02031598ea2aa689354c8f1a1ea35"} Jul 16 12:10:40.082272 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:40.082243 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-277mm" event={"ID":"f73a8a43-dd26-4e85-865c-a1aba38315c0","Type":"ContainerStarted","Data":"76625a816a4a4a90a6747db9a080c79b7b2c4a9c0122c1f5a5ab787f8f57834c"} Jul 16 12:10:40.083391 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:40.083363 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-72p8q" event={"ID":"c72b21b6-dfa9-49e6-b546-b0ac30e2216b","Type":"ContainerStarted","Data":"6f352d4675f36a13df6a4c6b5a5c5ca63fce0be9ce3b1fedce79a84a870ee4f6"} Jul 16 12:10:40.098350 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:40.098305 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-r97s6" podStartSLOduration=2.098288513 podStartE2EDuration="2.098288513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:10:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:10:40.09657515 +0000 UTC m=+189.859944614" watchObservedRunningTime="2026-07-16 12:10:40.098288513 +0000 UTC m=+189.861657976" Jul 16 12:10:40.473551 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:40.473233 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:40.473551 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:40.473343 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:40.473551 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:40.473389 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 12:10:40.473551 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:40.473483 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs podName:060e6722-ce48-468c-aa89-cc07918db901 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:42.473462659 +0000 UTC m=+192.236832104 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs") pod "router-default-68956567b4-ccm7k" (UID: "060e6722-ce48-468c-aa89-cc07918db901") : secret "router-metrics-certs-default" not found Jul 16 12:10:40.473551 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:40.473507 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle podName:060e6722-ce48-468c-aa89-cc07918db901 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:42.473494803 +0000 UTC m=+192.236864249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle") pod "router-default-68956567b4-ccm7k" (UID: "060e6722-ce48-468c-aa89-cc07918db901") : configmap references non-existent config key: service-ca.crt Jul 16 12:10:41.088621 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:41.088569 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c99fx" event={"ID":"1c7ed060-2647-40fb-b395-6654ac05cd42","Type":"ContainerStarted","Data":"f95d0fd8a1d5783727059b445509c6dfb639ebdb708512031043bd68303b0782"} Jul 16 12:10:41.110605 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:41.110540 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c99fx" podStartSLOduration=1.9478681660000001 podStartE2EDuration="3.110516439s" totalImagesPullingTime="1.162648273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:10:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:10:41.108115118 +0000 UTC m=+190.871484582" watchObservedRunningTime="2026-07-16 12:10:41.110516439 +0000 UTC m=+190.873885903" Jul 16 12:10:41.228044 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:41.228010 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:42.488670 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:42.488628 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:42.489076 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:42.488692 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:42.489076 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:42.488801 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 12:10:42.489076 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:42.488836 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle podName:060e6722-ce48-468c-aa89-cc07918db901 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:46.488812686 +0000 UTC m=+196.252182142 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle") pod "router-default-68956567b4-ccm7k" (UID: "060e6722-ce48-468c-aa89-cc07918db901") : configmap references non-existent config key: service-ca.crt Jul 16 12:10:42.489076 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:42.488859 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs podName:060e6722-ce48-468c-aa89-cc07918db901 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:46.488849953 +0000 UTC m=+196.252219395 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs") pod "router-default-68956567b4-ccm7k" (UID: "060e6722-ce48-468c-aa89-cc07918db901") : secret "router-metrics-certs-default" not found Jul 16 12:10:42.702850 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:42.702814 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:43.096780 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.096742 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd" event={"ID":"fdb05941-6cba-40f2-8150-7465f6b82a78","Type":"ContainerStarted","Data":"0d7d7ef7789652fd672a71da00df97c408a2b9a2d1e6e07eb6d7bef913d8fc12"} Jul 16 12:10:43.098384 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.098350 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-277mm" event={"ID":"f73a8a43-dd26-4e85-865c-a1aba38315c0","Type":"ContainerStarted","Data":"0e492905675c6663e71bf0c7e2db5fa34f35752da90ce7ef99a39e38ad6e991b"} Jul 16 12:10:43.118615 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.118562 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd" podStartSLOduration=2.269004032 podStartE2EDuration="5.118545136s" totalImagesPullingTime="2.849541104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:10:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:10:43.11772397 +0000 UTC m=+192.881093433" watchObservedRunningTime="2026-07-16 12:10:43.118545136 +0000 UTC m=+192.881914800" Jul 16 12:10:43.152056 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.151573 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-277mm" podStartSLOduration=2.304412941 podStartE2EDuration="5.151558541s" totalImagesPullingTime="2.8471456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:10:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:10:43.151456247 +0000 UTC m=+192.914825712" watchObservedRunningTime="2026-07-16 12:10:43.151558541 +0000 UTC m=+192.914928047" Jul 16 12:10:43.168474 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.168312 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:43.262140 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.262090 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:43.366574 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.366426 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:43.788090 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.788060 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-znnr6"] Jul 16 12:10:43.790154 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.790136 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:43.793745 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.793705 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 12:10:43.793745 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.793733 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8fngt\"" Jul 16 12:10:43.794474 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.794452 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 12:10:43.809273 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.809218 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-znnr6"] Jul 16 12:10:43.828905 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.828854 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:43.898868 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.898834 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:43.899055 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.898884 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-data-volume\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:43.899055 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.899001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:43.899155 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.899062 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-crio-socket\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:43.899155 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.899096 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td7wd\" (UniqueName: \"kubernetes.io/projected/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-api-access-td7wd\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:43.955576 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.955530 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:43.999793 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.999753 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:43.999966 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:43.999898 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 12:10:43.999966 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:43.999902 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-data-volume\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:44.000061 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:43.999989 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls podName:222026e5-8cb9-43d6-bbe0-25dcb7c66ce6 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:44.499963746 +0000 UTC m=+194.263333193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-znnr6" (UID: "222026e5-8cb9-43d6-bbe0-25dcb7c66ce6") : secret "insights-runtime-extractor-tls" not found Jul 16 12:10:44.000061 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:44.000038 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:44.000150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:44.000094 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-crio-socket\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:44.000150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:44.000123 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td7wd\" (UniqueName: \"kubernetes.io/projected/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-api-access-td7wd\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:44.000253 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:44.000163 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 12:10:44.000253 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:44.000201 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-crio-socket\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:44.000253 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:44.000233 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm podName:222026e5-8cb9-43d6-bbe0-25dcb7c66ce6 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:44.500214213 +0000 UTC m=+194.263583700 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-znnr6" (UID: "222026e5-8cb9-43d6-bbe0-25dcb7c66ce6") : configmap "kube-rbac-proxy" not found Jul 16 12:10:44.000397 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:44.000266 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-data-volume\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:44.012235 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:44.012203 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td7wd\" (UniqueName: \"kubernetes.io/projected/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-api-access-td7wd\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:44.505316 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:44.505277 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:44.505485 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:44.505349 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:44.505485 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:44.505444 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 12:10:44.505623 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:44.505507 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls podName:222026e5-8cb9-43d6-bbe0-25dcb7c66ce6 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:45.505490055 +0000 UTC m=+195.268859499 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-znnr6" (UID: "222026e5-8cb9-43d6-bbe0-25dcb7c66ce6") : secret "insights-runtime-extractor-tls" not found Jul 16 12:10:44.505623 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:44.505453 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 12:10:44.505623 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:44.505595 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm podName:222026e5-8cb9-43d6-bbe0-25dcb7c66ce6 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:45.505573924 +0000 UTC m=+195.268943370 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-znnr6" (UID: "222026e5-8cb9-43d6-bbe0-25dcb7c66ce6") : configmap "kube-rbac-proxy" not found Jul 16 12:10:44.798932 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:44.798897 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:44.869771 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:44.869732 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:45.107094 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:45.107022 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/0.log" Jul 16 12:10:45.107094 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:45.107058 2578 generic.go:356] "Generic (PLEG): container finished" podID="c72b21b6-dfa9-49e6-b546-b0ac30e2216b" containerID="366e7879b6fc7b65036332f5701ef251d1e473217c20f68b5419dccb79bdc19a" exitCode=255 Jul 16 12:10:45.107283 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:45.107102 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-72p8q" event={"ID":"c72b21b6-dfa9-49e6-b546-b0ac30e2216b","Type":"ContainerDied","Data":"366e7879b6fc7b65036332f5701ef251d1e473217c20f68b5419dccb79bdc19a"} Jul 16 12:10:45.107399 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:45.107377 2578 scope.go:117] "RemoveContainer" containerID="366e7879b6fc7b65036332f5701ef251d1e473217c20f68b5419dccb79bdc19a" Jul 16 12:10:45.445810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:45.445721 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:45.513557 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:45.513514 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:45.513557 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:45.513565 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:45.513841 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:45.513635 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 12:10:45.513841 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:45.513692 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 12:10:45.513841 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:45.513699 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm podName:222026e5-8cb9-43d6-bbe0-25dcb7c66ce6 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:47.513674907 +0000 UTC m=+197.277044349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-znnr6" (UID: "222026e5-8cb9-43d6-bbe0-25dcb7c66ce6") : configmap "kube-rbac-proxy" not found Jul 16 12:10:45.513841 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:45.513771 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls podName:222026e5-8cb9-43d6-bbe0-25dcb7c66ce6 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:47.513753697 +0000 UTC m=+197.277123150 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-znnr6" (UID: "222026e5-8cb9-43d6-bbe0-25dcb7c66ce6") : secret "insights-runtime-extractor-tls" not found Jul 16 12:10:45.640336 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:45.640300 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:45.745366 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:45.745271 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:45.946759 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:45.946732 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r7c42_e00ecaab-22f9-4338-ac72-f0eaad408347/dns-node-resolver/0.log" Jul 16 12:10:46.115030 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:46.115006 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:10:46.115500 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:46.115480 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/0.log" Jul 16 12:10:46.115594 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:46.115518 2578 generic.go:356] "Generic (PLEG): container finished" podID="c72b21b6-dfa9-49e6-b546-b0ac30e2216b" containerID="edf233af2357c11220d3c35baf389890b24fddd2118b22b28764c0cd2ea8ce2b" exitCode=255 Jul 16 12:10:46.115594 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:46.115549 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-72p8q" event={"ID":"c72b21b6-dfa9-49e6-b546-b0ac30e2216b","Type":"ContainerDied","Data":"edf233af2357c11220d3c35baf389890b24fddd2118b22b28764c0cd2ea8ce2b"} Jul 16 12:10:46.115663 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:46.115647 2578 scope.go:117] "RemoveContainer" containerID="366e7879b6fc7b65036332f5701ef251d1e473217c20f68b5419dccb79bdc19a" Jul 16 12:10:46.115843 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:46.115821 2578 scope.go:117] "RemoveContainer" containerID="edf233af2357c11220d3c35baf389890b24fddd2118b22b28764c0cd2ea8ce2b" Jul 16 12:10:46.116005 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:46.115981 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-72p8q_openshift-console-operator(c72b21b6-dfa9-49e6-b546-b0ac30e2216b)\"" pod="openshift-console-operator/console-operator-7866497494-72p8q" podUID="c72b21b6-dfa9-49e6-b546-b0ac30e2216b" Jul 16 12:10:46.188236 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:46.188205 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:46.312706 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:46.312669 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:46.482617 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:46.482519 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:46.522056 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:46.522022 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:46.522056 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:46.522059 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:46.522252 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:46.522170 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 12:10:46.522252 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:46.522191 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle podName:060e6722-ce48-468c-aa89-cc07918db901 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:54.522177511 +0000 UTC m=+204.285546952 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle") pod "router-default-68956567b4-ccm7k" (UID: "060e6722-ce48-468c-aa89-cc07918db901") : configmap references non-existent config key: service-ca.crt Jul 16 12:10:46.522252 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:46.522228 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs podName:060e6722-ce48-468c-aa89-cc07918db901 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:54.522214715 +0000 UTC m=+204.285584156 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs") pod "router-default-68956567b4-ccm7k" (UID: "060e6722-ce48-468c-aa89-cc07918db901") : secret "router-metrics-certs-default" not found Jul 16 12:10:46.599650 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:46.599612 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:46.699478 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:46.699441 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:46.800927 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:46.800895 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:46.945969 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:46.945941 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wjpmx_833615c3-9d95-448d-a709-4735c575a995/node-ca/0.log" Jul 16 12:10:47.120215 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:47.120139 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:10:47.120580 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:47.120491 2578 scope.go:117] "RemoveContainer" containerID="edf233af2357c11220d3c35baf389890b24fddd2118b22b28764c0cd2ea8ce2b" Jul 16 12:10:47.120676 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:47.120655 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-72p8q_openshift-console-operator(c72b21b6-dfa9-49e6-b546-b0ac30e2216b)\"" pod="openshift-console-operator/console-operator-7866497494-72p8q" podUID="c72b21b6-dfa9-49e6-b546-b0ac30e2216b" Jul 16 12:10:47.121769 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:47.121743 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:47.354016 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:47.353984 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-knfq6_10ea27e0-362a-4adb-a93c-e3db335e4610/serve-healthcheck-canary/0.log" Jul 16 12:10:47.528840 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:47.528744 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:47.529024 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:47.528893 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 12:10:47.529024 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:47.528948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:47.529024 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:47.528970 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm podName:222026e5-8cb9-43d6-bbe0-25dcb7c66ce6 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:51.528948384 +0000 UTC m=+201.292317844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-znnr6" (UID: "222026e5-8cb9-43d6-bbe0-25dcb7c66ce6") : configmap "kube-rbac-proxy" not found Jul 16 12:10:47.529149 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:47.529037 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 12:10:47.529149 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:47.529084 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls podName:222026e5-8cb9-43d6-bbe0-25dcb7c66ce6 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:51.529073148 +0000 UTC m=+201.292442594 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-znnr6" (UID: "222026e5-8cb9-43d6-bbe0-25dcb7c66ce6") : secret "insights-runtime-extractor-tls" not found Jul 16 12:10:47.819631 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:47.819597 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:48.211681 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:48.211592 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:48.609788 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:48.609732 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:48.710582 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:48.710538 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:49.016870 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:49.016783 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:49.052129 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:49.052098 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:10:49.052236 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:49.052166 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:10:49.052535 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:49.052521 2578 scope.go:117] "RemoveContainer" containerID="edf233af2357c11220d3c35baf389890b24fddd2118b22b28764c0cd2ea8ce2b" Jul 16 12:10:49.052694 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:49.052675 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-72p8q_openshift-console-operator(c72b21b6-dfa9-49e6-b546-b0ac30e2216b)\"" pod="openshift-console-operator/console-operator-7866497494-72p8q" podUID="c72b21b6-dfa9-49e6-b546-b0ac30e2216b" Jul 16 12:10:49.227199 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:49.227160 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:49.692585 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:49.692551 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:50.226969 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:50.226928 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:50.328258 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:50.328225 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:50.630669 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:50.630634 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:51.028020 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:51.027926 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:51.212830 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:51.212795 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:51.562679 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:51.562638 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:51.563166 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:51.562696 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:51.563166 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:51.562796 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 12:10:51.563166 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:51.562802 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 12:10:51.563166 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:51.562864 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm podName:222026e5-8cb9-43d6-bbe0-25dcb7c66ce6 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:59.562843671 +0000 UTC m=+209.326213144 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-znnr6" (UID: "222026e5-8cb9-43d6-bbe0-25dcb7c66ce6") : configmap "kube-rbac-proxy" not found Jul 16 12:10:51.563166 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:10:51.562883 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls podName:222026e5-8cb9-43d6-bbe0-25dcb7c66ce6 nodeName:}" failed. No retries permitted until 2026-07-16 12:10:59.562874888 +0000 UTC m=+209.326244329 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-znnr6" (UID: "222026e5-8cb9-43d6-bbe0-25dcb7c66ce6") : secret "insights-runtime-extractor-tls" not found Jul 16 12:10:51.839211 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:51.839113 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:52.423336 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:52.423299 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:52.503477 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:52.503426 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:52.633933 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:52.633885 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:52.821995 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:52.821963 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:53.215987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:53.215895 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:53.834598 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:53.834558 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:54.211077 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:54.210974 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:54.591377 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:54.591338 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:54.591377 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:54.591377 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:54.591980 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:54.591958 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/060e6722-ce48-468c-aa89-cc07918db901-service-ca-bundle\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:54.593852 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:54.593822 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/060e6722-ce48-468c-aa89-cc07918db901-metrics-certs\") pod \"router-default-68956567b4-ccm7k\" (UID: \"060e6722-ce48-468c-aa89-cc07918db901\") " pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:54.608279 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:54.608247 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:54.887678 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:54.887593 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:55.039636 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:55.039607 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68956567b4-ccm7k"] Jul 16 12:10:55.044314 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:10:55.044283 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod060e6722_ce48_468c_aa89_cc07918db901.slice/crio-10ce94c9be32af995a9d5523a58f9757b59a2e1caaeac2c50ca8983ba62d1079 WatchSource:0}: Error finding container 10ce94c9be32af995a9d5523a58f9757b59a2e1caaeac2c50ca8983ba62d1079: Status 404 returned error can't find the container with id 10ce94c9be32af995a9d5523a58f9757b59a2e1caaeac2c50ca8983ba62d1079 Jul 16 12:10:55.086086 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:55.086058 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:55.146076 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:55.145997 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68956567b4-ccm7k" event={"ID":"060e6722-ce48-468c-aa89-cc07918db901","Type":"ContainerStarted","Data":"b896100863850ed82ba0f1d3dba439cee5525c7b0d10e0c648d2711a55ec1076"} Jul 16 12:10:55.146076 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:55.146054 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68956567b4-ccm7k" event={"ID":"060e6722-ce48-468c-aa89-cc07918db901","Type":"ContainerStarted","Data":"10ce94c9be32af995a9d5523a58f9757b59a2e1caaeac2c50ca8983ba62d1079"} Jul 16 12:10:55.174168 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:55.174118 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-68956567b4-ccm7k" podStartSLOduration=17.174100764 podStartE2EDuration="17.174100764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:10:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:10:55.173703439 +0000 UTC m=+204.937072926" watchObservedRunningTime="2026-07-16 12:10:55.174100764 +0000 UTC m=+204.937470227" Jul 16 12:10:55.405849 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:55.405758 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:55.613325 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:55.613286 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:55.762522 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:55.762444 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:55.870039 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:55.870007 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:55.888010 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:55.887983 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:55.890565 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:55.890544 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:56.149493 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:56.149464 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:56.150715 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:56.150690 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-68956567b4-ccm7k" Jul 16 12:10:56.438147 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:56.438046 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:56.609239 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:56.609206 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:57.059731 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:57.059697 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:57.254008 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:57.253970 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:57.531959 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:57.531930 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:57.845177 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:57.845080 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:58.028785 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:58.028731 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:58.819812 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:58.819779 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:58.902735 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:58.902702 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:10:59.634067 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:59.634027 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:59.634280 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:59.634081 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:59.634688 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:59.634666 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:59.636439 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:59.636396 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/222026e5-8cb9-43d6-bbe0-25dcb7c66ce6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-znnr6\" (UID: \"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6\") " pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:59.701518 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:59.701482 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-znnr6" Jul 16 12:10:59.846394 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:10:59.846368 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-znnr6"] Jul 16 12:10:59.848691 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:10:59.848664 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod222026e5_8cb9_43d6_bbe0_25dcb7c66ce6.slice/crio-450a869fe053282be2556b497cdf23f072e96d4d1d9c4d87dd22887310232240 WatchSource:0}: Error finding container 450a869fe053282be2556b497cdf23f072e96d4d1d9c4d87dd22887310232240: Status 404 returned error can't find the container with id 450a869fe053282be2556b497cdf23f072e96d4d1d9c4d87dd22887310232240 Jul 16 12:11:00.162264 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:00.162230 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-znnr6" event={"ID":"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6","Type":"ContainerStarted","Data":"450a869fe053282be2556b497cdf23f072e96d4d1d9c4d87dd22887310232240"} Jul 16 12:11:03.473681 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:03.473637 2578 scope.go:117] "RemoveContainer" containerID="edf233af2357c11220d3c35baf389890b24fddd2118b22b28764c0cd2ea8ce2b" Jul 16 12:11:05.181492 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:05.181450 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-znnr6" event={"ID":"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6","Type":"ContainerStarted","Data":"a8770fb833068b888b51da89f2f8b03f88cbdf8c7b0ac8e42edfbcd9e6044626"} Jul 16 12:11:05.183604 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:05.183573 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:11:05.183753 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:05.183656 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-72p8q" event={"ID":"c72b21b6-dfa9-49e6-b546-b0ac30e2216b","Type":"ContainerStarted","Data":"025d64626d2eef063df3bc1eb7226d8a74c7575d3b581f078a6d8224f3b7b17d"} Jul 16 12:11:05.184024 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:05.184003 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:11:05.210400 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:05.210338 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-72p8q" podStartSLOduration=22.011439888 podStartE2EDuration="27.210316065s" totalImagesPullingTime="5.198876177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:10:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:11:05.208992223 +0000 UTC m=+214.972361712" watchObservedRunningTime="2026-07-16 12:11:05.210316065 +0000 UTC m=+214.973685509" Jul 16 12:11:05.780223 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:05.780192 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-72p8q" Jul 16 12:11:06.009980 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:06.009943 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:06.114862 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:06.114756 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:06.278618 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:06.278565 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:06.657238 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:06.657207 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:06.770752 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:06.770713 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:07.070384 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:07.070344 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:07.815104 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:07.815064 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:08.014051 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.014012 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:08.078336 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.078263 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-pvgch"] Jul 16 12:11:08.081396 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.081373 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-pvgch" Jul 16 12:11:08.083862 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.083838 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 12:11:08.085970 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.085948 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-6qhlv\"" Jul 16 12:11:08.098101 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.098072 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 12:11:08.123207 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.123177 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4vxgp"] Jul 16 12:11:08.127981 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.127951 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-pvgch"] Jul 16 12:11:08.128130 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.128092 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4vxgp" Jul 16 12:11:08.133998 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.133939 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-8cxc8\"" Jul 16 12:11:08.133998 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.133993 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 16 12:11:08.192729 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.192698 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4vxgp"] Jul 16 12:11:08.204306 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.204269 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/28f37fb8-7bc6-4d74-a948-b39a0b259637-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-4vxgp\" (UID: \"28f37fb8-7bc6-4d74-a948-b39a0b259637\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4vxgp" Jul 16 12:11:08.204468 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.204380 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg6jj\" (UniqueName: \"kubernetes.io/projected/da0d38c7-23cc-4057-bb7a-0de38b2ee970-kube-api-access-wg6jj\") pod \"downloads-68c9cbb497-pvgch\" (UID: \"da0d38c7-23cc-4057-bb7a-0de38b2ee970\") " pod="openshift-console/downloads-68c9cbb497-pvgch" Jul 16 12:11:08.228923 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.228891 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:08.305279 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.305249 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/28f37fb8-7bc6-4d74-a948-b39a0b259637-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-4vxgp\" (UID: \"28f37fb8-7bc6-4d74-a948-b39a0b259637\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4vxgp" Jul 16 12:11:08.305438 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.305316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg6jj\" (UniqueName: \"kubernetes.io/projected/da0d38c7-23cc-4057-bb7a-0de38b2ee970-kube-api-access-wg6jj\") pod \"downloads-68c9cbb497-pvgch\" (UID: \"da0d38c7-23cc-4057-bb7a-0de38b2ee970\") " pod="openshift-console/downloads-68c9cbb497-pvgch" Jul 16 12:11:08.307614 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.307587 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/28f37fb8-7bc6-4d74-a948-b39a0b259637-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-4vxgp\" (UID: \"28f37fb8-7bc6-4d74-a948-b39a0b259637\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4vxgp" Jul 16 12:11:08.369465 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.369389 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg6jj\" (UniqueName: \"kubernetes.io/projected/da0d38c7-23cc-4057-bb7a-0de38b2ee970-kube-api-access-wg6jj\") pod \"downloads-68c9cbb497-pvgch\" (UID: \"da0d38c7-23cc-4057-bb7a-0de38b2ee970\") " pod="openshift-console/downloads-68c9cbb497-pvgch" Jul 16 12:11:08.392127 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.392096 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-pvgch" Jul 16 12:11:08.422874 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.422842 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64d797f876-5qg7q"] Jul 16 12:11:08.427021 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.426995 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.431495 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.431469 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 12:11:08.431992 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.431961 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-knmbz\"" Jul 16 12:11:08.431992 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.431988 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 12:11:08.433143 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.433121 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 12:11:08.441711 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.441604 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4vxgp" Jul 16 12:11:08.467702 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.467668 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 12:11:08.488780 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.488733 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64d797f876-5qg7q"] Jul 16 12:11:08.506952 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.506915 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8s2c\" (UniqueName: \"kubernetes.io/projected/7331897a-45b7-4b04-be5c-9bb16e47b9a9-kube-api-access-t8s2c\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.507104 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.506984 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7331897a-45b7-4b04-be5c-9bb16e47b9a9-ca-trust-extracted\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.507104 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.507025 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7331897a-45b7-4b04-be5c-9bb16e47b9a9-trusted-ca\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.507104 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.507058 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7331897a-45b7-4b04-be5c-9bb16e47b9a9-image-registry-private-configuration\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.507104 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.507095 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7331897a-45b7-4b04-be5c-9bb16e47b9a9-installation-pull-secrets\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.507335 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.507119 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7331897a-45b7-4b04-be5c-9bb16e47b9a9-bound-sa-token\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.507335 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.507146 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7331897a-45b7-4b04-be5c-9bb16e47b9a9-registry-tls\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.507335 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.507175 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7331897a-45b7-4b04-be5c-9bb16e47b9a9-registry-certificates\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.607861 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.607828 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8s2c\" (UniqueName: \"kubernetes.io/projected/7331897a-45b7-4b04-be5c-9bb16e47b9a9-kube-api-access-t8s2c\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.608027 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.607881 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7331897a-45b7-4b04-be5c-9bb16e47b9a9-ca-trust-extracted\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.608027 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.607908 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7331897a-45b7-4b04-be5c-9bb16e47b9a9-trusted-ca\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.608027 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.607928 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7331897a-45b7-4b04-be5c-9bb16e47b9a9-image-registry-private-configuration\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.608027 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.607950 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7331897a-45b7-4b04-be5c-9bb16e47b9a9-installation-pull-secrets\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.608027 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.607966 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7331897a-45b7-4b04-be5c-9bb16e47b9a9-bound-sa-token\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.608027 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.607982 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7331897a-45b7-4b04-be5c-9bb16e47b9a9-registry-tls\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.608027 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.608012 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7331897a-45b7-4b04-be5c-9bb16e47b9a9-registry-certificates\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.608533 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.608510 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7331897a-45b7-4b04-be5c-9bb16e47b9a9-ca-trust-extracted\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.608966 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.608945 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7331897a-45b7-4b04-be5c-9bb16e47b9a9-trusted-ca\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.609064 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.609002 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7331897a-45b7-4b04-be5c-9bb16e47b9a9-registry-certificates\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.610547 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.610524 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7331897a-45b7-4b04-be5c-9bb16e47b9a9-installation-pull-secrets\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.610627 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.610548 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7331897a-45b7-4b04-be5c-9bb16e47b9a9-image-registry-private-configuration\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.610627 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.610577 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7331897a-45b7-4b04-be5c-9bb16e47b9a9-registry-tls\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.617342 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.617323 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-pvgch"] Jul 16 12:11:08.620472 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:11:08.620392 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda0d38c7_23cc_4057_bb7a_0de38b2ee970.slice/crio-23b9dcd640e2eecbf29b08f641ebaa06303909d6e1a1108f1cdaa27a6f1b20be WatchSource:0}: Error finding container 23b9dcd640e2eecbf29b08f641ebaa06303909d6e1a1108f1cdaa27a6f1b20be: Status 404 returned error can't find the container with id 23b9dcd640e2eecbf29b08f641ebaa06303909d6e1a1108f1cdaa27a6f1b20be Jul 16 12:11:08.642336 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.642310 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7331897a-45b7-4b04-be5c-9bb16e47b9a9-bound-sa-token\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.648115 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.648088 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4vxgp"] Jul 16 12:11:08.648573 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.648550 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8s2c\" (UniqueName: \"kubernetes.io/projected/7331897a-45b7-4b04-be5c-9bb16e47b9a9-kube-api-access-t8s2c\") pod \"image-registry-64d797f876-5qg7q\" (UID: \"7331897a-45b7-4b04-be5c-9bb16e47b9a9\") " pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.651325 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:11:08.651300 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f37fb8_7bc6_4d74_a948_b39a0b259637.slice/crio-4f9ea15ef31307ad03f1828160045637082fdfa306c88df2c06cebbcf776daf1 WatchSource:0}: Error finding container 4f9ea15ef31307ad03f1828160045637082fdfa306c88df2c06cebbcf776daf1: Status 404 returned error can't find the container with id 4f9ea15ef31307ad03f1828160045637082fdfa306c88df2c06cebbcf776daf1 Jul 16 12:11:08.737115 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.737075 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:08.925669 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:08.925575 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64d797f876-5qg7q"] Jul 16 12:11:08.928718 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:11:08.928681 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7331897a_45b7_4b04_be5c_9bb16e47b9a9.slice/crio-5cd6f7242ba73f8bf147716ada4ecd87a2df1f8adc88fee834789693add54422 WatchSource:0}: Error finding container 5cd6f7242ba73f8bf147716ada4ecd87a2df1f8adc88fee834789693add54422: Status 404 returned error can't find the container with id 5cd6f7242ba73f8bf147716ada4ecd87a2df1f8adc88fee834789693add54422 Jul 16 12:11:09.199103 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:09.199002 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64d797f876-5qg7q" event={"ID":"7331897a-45b7-4b04-be5c-9bb16e47b9a9","Type":"ContainerStarted","Data":"dcc05f29594aa89cc26ae0c69462262d8a2bb702c0a54063a111db12d0359560"} Jul 16 12:11:09.199103 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:09.199074 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64d797f876-5qg7q" event={"ID":"7331897a-45b7-4b04-be5c-9bb16e47b9a9","Type":"ContainerStarted","Data":"5cd6f7242ba73f8bf147716ada4ecd87a2df1f8adc88fee834789693add54422"} Jul 16 12:11:09.199337 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:09.199215 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:09.200468 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:09.200442 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-pvgch" event={"ID":"da0d38c7-23cc-4057-bb7a-0de38b2ee970","Type":"ContainerStarted","Data":"23b9dcd640e2eecbf29b08f641ebaa06303909d6e1a1108f1cdaa27a6f1b20be"} Jul 16 12:11:09.202724 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:09.202697 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-znnr6" event={"ID":"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6","Type":"ContainerStarted","Data":"ea9712988067c126057e0d8a88105a56ea9f7c3b91434a964fa5156a24a734e4"} Jul 16 12:11:09.204138 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:09.204114 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4vxgp" event={"ID":"28f37fb8-7bc6-4d74-a948-b39a0b259637","Type":"ContainerStarted","Data":"4f9ea15ef31307ad03f1828160045637082fdfa306c88df2c06cebbcf776daf1"} Jul 16 12:11:10.210180 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:10.210141 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4vxgp" event={"ID":"28f37fb8-7bc6-4d74-a948-b39a0b259637","Type":"ContainerStarted","Data":"a3192d03c6503122e6a0a37745be90026545f5c6cc736760e562df8a10c6a98b"} Jul 16 12:11:10.233561 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:10.233472 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4vxgp" podStartSLOduration=0.837044821 podStartE2EDuration="2.23292726s" totalImagesPullingTime="1.395882439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:11:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:11:10.232369815 +0000 UTC m=+219.995739278" watchObservedRunningTime="2026-07-16 12:11:10.23292726 +0000 UTC m=+219.996296703" Jul 16 12:11:10.233761 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:10.233650 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64d797f876-5qg7q" podStartSLOduration=2.233634728 podStartE2EDuration="2.233634728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:11:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:11:09.280341997 +0000 UTC m=+219.043711461" watchObservedRunningTime="2026-07-16 12:11:10.233634728 +0000 UTC m=+219.997004192" Jul 16 12:11:10.674364 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:10.674236 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:10.760057 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:10.760020 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:11.214743 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.214709 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4vxgp" Jul 16 12:11:11.220674 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.220646 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-4vxgp" Jul 16 12:11:11.624870 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.624832 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:11.721373 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.721298 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-89llm"] Jul 16 12:11:11.738870 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.738808 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:11.744964 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.744884 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-89llm"] Jul 16 12:11:11.744964 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.744920 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" Jul 16 12:11:11.748781 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.748759 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 16 12:11:11.748781 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.748775 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 16 12:11:11.748971 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.748786 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-pdhsl\"" Jul 16 12:11:11.749651 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.749633 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 12:11:11.749811 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.749634 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 12:11:11.750451 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.750430 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 12:11:11.838884 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.838841 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhl6t\" (UniqueName: \"kubernetes.io/projected/83e1ab03-695b-4979-8df4-7b647f11b770-kube-api-access-hhl6t\") pod \"prometheus-operator-64785b6675-89llm\" (UID: \"83e1ab03-695b-4979-8df4-7b647f11b770\") " pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" Jul 16 12:11:11.839066 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.838933 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83e1ab03-695b-4979-8df4-7b647f11b770-metrics-client-ca\") pod \"prometheus-operator-64785b6675-89llm\" (UID: \"83e1ab03-695b-4979-8df4-7b647f11b770\") " pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" Jul 16 12:11:11.839066 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.839016 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/83e1ab03-695b-4979-8df4-7b647f11b770-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-89llm\" (UID: \"83e1ab03-695b-4979-8df4-7b647f11b770\") " pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" Jul 16 12:11:11.839173 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.839078 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83e1ab03-695b-4979-8df4-7b647f11b770-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-89llm\" (UID: \"83e1ab03-695b-4979-8df4-7b647f11b770\") " pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" Jul 16 12:11:11.934636 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.934459 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:11.939788 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.939756 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83e1ab03-695b-4979-8df4-7b647f11b770-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-89llm\" (UID: \"83e1ab03-695b-4979-8df4-7b647f11b770\") " pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" Jul 16 12:11:11.939942 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.939852 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhl6t\" (UniqueName: \"kubernetes.io/projected/83e1ab03-695b-4979-8df4-7b647f11b770-kube-api-access-hhl6t\") pod \"prometheus-operator-64785b6675-89llm\" (UID: \"83e1ab03-695b-4979-8df4-7b647f11b770\") " pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" Jul 16 12:11:11.939942 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.939882 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83e1ab03-695b-4979-8df4-7b647f11b770-metrics-client-ca\") pod \"prometheus-operator-64785b6675-89llm\" (UID: \"83e1ab03-695b-4979-8df4-7b647f11b770\") " pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" Jul 16 12:11:11.939942 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.939935 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/83e1ab03-695b-4979-8df4-7b647f11b770-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-89llm\" (UID: \"83e1ab03-695b-4979-8df4-7b647f11b770\") " pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" Jul 16 12:11:11.941029 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.940988 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83e1ab03-695b-4979-8df4-7b647f11b770-metrics-client-ca\") pod \"prometheus-operator-64785b6675-89llm\" (UID: \"83e1ab03-695b-4979-8df4-7b647f11b770\") " pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" Jul 16 12:11:11.943329 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.943302 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/83e1ab03-695b-4979-8df4-7b647f11b770-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-89llm\" (UID: \"83e1ab03-695b-4979-8df4-7b647f11b770\") " pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" Jul 16 12:11:11.943483 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.943336 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83e1ab03-695b-4979-8df4-7b647f11b770-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-89llm\" (UID: \"83e1ab03-695b-4979-8df4-7b647f11b770\") " pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" Jul 16 12:11:11.955153 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:11.955119 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhl6t\" (UniqueName: \"kubernetes.io/projected/83e1ab03-695b-4979-8df4-7b647f11b770-kube-api-access-hhl6t\") pod \"prometheus-operator-64785b6675-89llm\" (UID: \"83e1ab03-695b-4979-8df4-7b647f11b770\") " pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" Jul 16 12:11:12.057239 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:12.057164 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" Jul 16 12:11:12.301185 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:12.301145 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-89llm"] Jul 16 12:11:12.305229 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:11:12.305198 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83e1ab03_695b_4979_8df4_7b647f11b770.slice/crio-d59de90aae74b23d362889663fa3edc7da416a42fa7824a6e416cce6d59b8d0b WatchSource:0}: Error finding container d59de90aae74b23d362889663fa3edc7da416a42fa7824a6e416cce6d59b8d0b: Status 404 returned error can't find the container with id d59de90aae74b23d362889663fa3edc7da416a42fa7824a6e416cce6d59b8d0b Jul 16 12:11:12.438739 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:12.438697 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:12.541291 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:12.541246 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:13.225006 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:13.224910 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" event={"ID":"83e1ab03-695b-4979-8df4-7b647f11b770","Type":"ContainerStarted","Data":"d59de90aae74b23d362889663fa3edc7da416a42fa7824a6e416cce6d59b8d0b"} Jul 16 12:11:13.227503 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:13.227461 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-znnr6" event={"ID":"222026e5-8cb9-43d6-bbe0-25dcb7c66ce6","Type":"ContainerStarted","Data":"df5f0a5490b93ce31b3beca2690eaed154c09253063fe349bc379609a6371482"} Jul 16 12:11:13.263040 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:13.262980 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-znnr6" podStartSLOduration=18.035041631 podStartE2EDuration="30.262958367s" totalImagesPullingTime="12.227916736s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:10:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:11:13.262295354 +0000 UTC m=+223.025664830" watchObservedRunningTime="2026-07-16 12:11:13.262958367 +0000 UTC m=+223.026327833" Jul 16 12:11:14.221004 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:14.220891 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:14.233809 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:14.233760 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" event={"ID":"83e1ab03-695b-4979-8df4-7b647f11b770","Type":"ContainerStarted","Data":"56af56053350fa428f2e360a46d8980f4a3805a6220d81afbae4c6746a02fcb3"} Jul 16 12:11:14.233980 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:14.233827 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" event={"ID":"83e1ab03-695b-4979-8df4-7b647f11b770","Type":"ContainerStarted","Data":"75800fc087f16ccfed163ac89c772842a3945d348c7219948553c29164db89cd"} Jul 16 12:11:15.988307 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:15.988271 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:16.343337 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.343296 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:16.542472 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.542429 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:16.758537 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.758395 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:16.759550 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.759502 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-89llm" podStartSLOduration=4.247427637 podStartE2EDuration="5.759485102s" totalImagesPullingTime="1.512057465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:11:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:11:14.259731867 +0000 UTC m=+224.023101331" watchObservedRunningTime="2026-07-16 12:11:16.759485102 +0000 UTC m=+226.522854565" Jul 16 12:11:16.760280 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.760259 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nmpvs"] Jul 16 12:11:16.779684 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.779645 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.789884 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.789826 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wcl2h\"" Jul 16 12:11:16.790034 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.789829 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 12:11:16.790169 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.790154 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 12:11:16.791000 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.790982 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 12:11:16.884754 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.884716 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7601f52b-2d35-4c6c-a682-719c85050b81-root\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.884916 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.884770 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7601f52b-2d35-4c6c-a682-719c85050b81-node-exporter-textfile\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.884916 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.884888 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7601f52b-2d35-4c6c-a682-719c85050b81-node-exporter-tls\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.885046 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.884923 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7601f52b-2d35-4c6c-a682-719c85050b81-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.885046 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.884954 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7601f52b-2d35-4c6c-a682-719c85050b81-node-exporter-accelerators-collector-config\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.885046 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.884993 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7601f52b-2d35-4c6c-a682-719c85050b81-node-exporter-wtmp\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.885183 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.885060 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7601f52b-2d35-4c6c-a682-719c85050b81-metrics-client-ca\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.885183 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.885092 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7601f52b-2d35-4c6c-a682-719c85050b81-sys\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.885183 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.885160 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z92wp\" (UniqueName: \"kubernetes.io/projected/7601f52b-2d35-4c6c-a682-719c85050b81-kube-api-access-z92wp\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.903158 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.903119 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:16.986258 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.986208 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7601f52b-2d35-4c6c-a682-719c85050b81-node-exporter-wtmp\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.986258 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.986264 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7601f52b-2d35-4c6c-a682-719c85050b81-metrics-client-ca\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.986533 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.986286 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7601f52b-2d35-4c6c-a682-719c85050b81-sys\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.986533 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.986317 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z92wp\" (UniqueName: \"kubernetes.io/projected/7601f52b-2d35-4c6c-a682-719c85050b81-kube-api-access-z92wp\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.986533 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.986385 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7601f52b-2d35-4c6c-a682-719c85050b81-root\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.986533 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.986425 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7601f52b-2d35-4c6c-a682-719c85050b81-node-exporter-textfile\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.986533 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.986438 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7601f52b-2d35-4c6c-a682-719c85050b81-sys\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.986533 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.986451 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7601f52b-2d35-4c6c-a682-719c85050b81-node-exporter-wtmp\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.986533 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.986505 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7601f52b-2d35-4c6c-a682-719c85050b81-node-exporter-tls\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.986533 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.986534 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7601f52b-2d35-4c6c-a682-719c85050b81-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.986880 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.986575 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7601f52b-2d35-4c6c-a682-719c85050b81-node-exporter-accelerators-collector-config\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.986880 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.986510 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7601f52b-2d35-4c6c-a682-719c85050b81-root\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.986880 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.986781 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7601f52b-2d35-4c6c-a682-719c85050b81-node-exporter-textfile\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.986993 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.986963 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7601f52b-2d35-4c6c-a682-719c85050b81-metrics-client-ca\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.987237 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.987210 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7601f52b-2d35-4c6c-a682-719c85050b81-node-exporter-accelerators-collector-config\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.989267 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.989240 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7601f52b-2d35-4c6c-a682-719c85050b81-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:16.989667 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:16.989481 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7601f52b-2d35-4c6c-a682-719c85050b81-node-exporter-tls\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:17.011738 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.011670 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z92wp\" (UniqueName: \"kubernetes.io/projected/7601f52b-2d35-4c6c-a682-719c85050b81-kube-api-access-z92wp\") pod \"node-exporter-nmpvs\" (UID: \"7601f52b-2d35-4c6c-a682-719c85050b81\") " pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:17.090557 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.090519 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nmpvs" Jul 16 12:11:17.103153 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.103078 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:17.119957 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:11:17.119923 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7601f52b_2d35_4c6c_a682_719c85050b81.slice/crio-373b699c2fac5029d8e82d8c7cf4b3e69a69b6df3eb8d9ab637c75f7a9fb19bb WatchSource:0}: Error finding container 373b699c2fac5029d8e82d8c7cf4b3e69a69b6df3eb8d9ab637c75f7a9fb19bb: Status 404 returned error can't find the container with id 373b699c2fac5029d8e82d8c7cf4b3e69a69b6df3eb8d9ab637c75f7a9fb19bb Jul 16 12:11:17.246490 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.246448 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nmpvs" event={"ID":"7601f52b-2d35-4c6c-a682-719c85050b81","Type":"ContainerStarted","Data":"373b699c2fac5029d8e82d8c7cf4b3e69a69b6df3eb8d9ab637c75f7a9fb19bb"} Jul 16 12:11:17.845813 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.845772 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 12:11:17.849218 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.849193 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:17.854320 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.854173 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 16 12:11:17.854320 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.854276 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 16 12:11:17.854534 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.854203 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 16 12:11:17.854534 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.854399 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 16 12:11:17.855727 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.854944 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 16 12:11:17.855727 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.855015 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 16 12:11:17.855727 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.855169 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-p2pjx\"" Jul 16 12:11:17.855727 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.855235 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 16 12:11:17.855727 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.855590 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 16 12:11:17.856556 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.856507 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 16 12:11:17.881245 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.881102 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 12:11:17.995822 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.995777 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:17.995822 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.995818 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-tls-assets\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:17.996325 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.995848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-web-config\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:17.996325 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.995877 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:17.996325 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.995926 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:17.996325 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.995950 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmsvp\" (UniqueName: \"kubernetes.io/projected/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-kube-api-access-zmsvp\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:17.996325 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.995981 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:17.996325 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.996006 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-config-volume\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:17.996325 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.996058 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:17.996325 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.996096 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:17.996325 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.996126 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:17.996325 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.996167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:17.996325 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:17.996188 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-config-out\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.097810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.097706 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.097810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.097774 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.097810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.097806 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-config-out\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.098087 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.097849 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.098087 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.097881 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-tls-assets\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.098087 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.097912 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-web-config\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.098087 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.097940 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.098087 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.097999 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.098087 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.098023 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmsvp\" (UniqueName: \"kubernetes.io/projected/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-kube-api-access-zmsvp\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.098087 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.098057 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.098087 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.098083 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-config-volume\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.098512 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.098130 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.098512 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.098162 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.099163 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.099130 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.099940 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.099912 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.100910 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.100886 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.101729 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.101687 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.102463 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.102251 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.102463 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.102332 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-web-config\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.104189 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.102630 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.104189 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.102956 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-config-out\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.104189 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.103474 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-tls-assets\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.104189 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.104181 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.104404 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.104257 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.105340 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.105315 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-config-volume\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.122739 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.122688 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmsvp\" (UniqueName: \"kubernetes.io/projected/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-kube-api-access-zmsvp\") pod \"alertmanager-main-0\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.163511 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.163469 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:11:18.452687 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:11:18.452653 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a3e2f1_66bb_41f2_afe1_79c2b8e67595.slice/crio-44d84eee6e2646f2f90b8d3649627fa35589bbe5dc71db1aaad64949ad379dce WatchSource:0}: Error finding container 44d84eee6e2646f2f90b8d3649627fa35589bbe5dc71db1aaad64949ad379dce: Status 404 returned error can't find the container with id 44d84eee6e2646f2f90b8d3649627fa35589bbe5dc71db1aaad64949ad379dce Jul 16 12:11:18.455868 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:18.455761 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 12:11:19.256200 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.256161 2578 generic.go:356] "Generic (PLEG): container finished" podID="7601f52b-2d35-4c6c-a682-719c85050b81" containerID="3965862684adf0eddf55bba798a34f0b4bca69c78699ea31eb9c20d0aa493e9c" exitCode=0 Jul 16 12:11:19.256691 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.256256 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nmpvs" event={"ID":"7601f52b-2d35-4c6c-a682-719c85050b81","Type":"ContainerDied","Data":"3965862684adf0eddf55bba798a34f0b4bca69c78699ea31eb9c20d0aa493e9c"} Jul 16 12:11:19.257718 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.257685 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28a3e2f1-66bb-41f2-afe1-79c2b8e67595","Type":"ContainerStarted","Data":"44d84eee6e2646f2f90b8d3649627fa35589bbe5dc71db1aaad64949ad379dce"} Jul 16 12:11:19.531294 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.531251 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:19.676797 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.676763 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:19.759947 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.759911 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b8bc7b8cd-sdr5h"] Jul 16 12:11:19.762685 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.762660 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:19.767288 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.767263 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-b64mj\"" Jul 16 12:11:19.768073 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.768047 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 12:11:19.768188 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.768111 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 12:11:19.768252 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.768112 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 12:11:19.769256 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.769235 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 12:11:19.770287 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.770098 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 12:11:19.775673 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.775654 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 12:11:19.798973 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.798896 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b8bc7b8cd-sdr5h"] Jul 16 12:11:19.833339 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.833288 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:19.915156 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.915124 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/355b8eed-b03b-4f9f-8add-9163af2a965f-console-oauth-config\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:19.915319 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.915186 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-trusted-ca-bundle\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:19.915319 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.915228 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqf5x\" (UniqueName: \"kubernetes.io/projected/355b8eed-b03b-4f9f-8add-9163af2a965f-kube-api-access-lqf5x\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:19.915319 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.915289 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-oauth-serving-cert\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:19.915512 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.915327 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-console-config\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:19.915512 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.915349 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-service-ca\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:19.915512 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:19.915375 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/355b8eed-b03b-4f9f-8add-9163af2a965f-console-serving-cert\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:20.016292 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.016258 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-service-ca\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:20.016499 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.016316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/355b8eed-b03b-4f9f-8add-9163af2a965f-console-serving-cert\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:20.016499 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.016375 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/355b8eed-b03b-4f9f-8add-9163af2a965f-console-oauth-config\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:20.016499 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.016472 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-trusted-ca-bundle\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:20.016680 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.016505 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqf5x\" (UniqueName: \"kubernetes.io/projected/355b8eed-b03b-4f9f-8add-9163af2a965f-kube-api-access-lqf5x\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:20.016680 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.016559 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-oauth-serving-cert\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:20.016680 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.016602 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-console-config\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:20.017145 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.017116 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-service-ca\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:20.017278 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.017269 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-console-config\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:20.017719 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.017695 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-trusted-ca-bundle\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:20.018002 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.017977 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-oauth-serving-cert\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:20.019559 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.019537 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/355b8eed-b03b-4f9f-8add-9163af2a965f-console-oauth-config\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:20.019685 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.019664 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/355b8eed-b03b-4f9f-8add-9163af2a965f-console-serving-cert\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:20.034841 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.034804 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqf5x\" (UniqueName: \"kubernetes.io/projected/355b8eed-b03b-4f9f-8add-9163af2a965f-kube-api-access-lqf5x\") pod \"console-7b8bc7b8cd-sdr5h\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:20.074549 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.074469 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:20.090464 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.090426 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:20.317947 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.317907 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:20.936038 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.935999 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-58d688cdfc-wrpdp"] Jul 16 12:11:20.939342 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.939310 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:20.944317 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.944270 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 16 12:11:20.950563 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.950439 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-4s6jg\"" Jul 16 12:11:20.950563 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.950467 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-60032b1kiteul\"" Jul 16 12:11:20.950563 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.950493 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 16 12:11:20.950563 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.950518 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 16 12:11:20.950563 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.950472 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 16 12:11:20.950869 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.950823 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 16 12:11:20.968304 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:20.968270 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-58d688cdfc-wrpdp"] Jul 16 12:11:21.030145 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.030107 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:21.076999 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.076962 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x"] Jul 16 12:11:21.079391 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.079369 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x" Jul 16 12:11:21.086431 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.086351 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-4znfs\"" Jul 16 12:11:21.086567 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.086537 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 16 12:11:21.104449 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.104424 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x"] Jul 16 12:11:21.127692 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.127659 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-thanos-querier-tls\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.127692 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.127693 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.127924 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.127720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.127924 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.127745 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.127924 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.127832 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5091f4e3-046e-4fe9-a173-976eb62cbd62-metrics-client-ca\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.128080 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.127930 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4kvr\" (UniqueName: \"kubernetes.io/projected/5091f4e3-046e-4fe9-a173-976eb62cbd62-kube-api-access-d4kvr\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.128080 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.127987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.128080 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.128052 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-grpc-tls\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.142285 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.142252 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:21.229558 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.229459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-thanos-querier-tls\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.229558 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.229500 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.229558 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.229527 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.229814 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.229634 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.229814 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.229707 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5091f4e3-046e-4fe9-a173-976eb62cbd62-metrics-client-ca\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.229814 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.229760 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4kvr\" (UniqueName: \"kubernetes.io/projected/5091f4e3-046e-4fe9-a173-976eb62cbd62-kube-api-access-d4kvr\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.229814 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.229797 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/27006609-ebbc-4519-a04e-0785af6dadbd-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-5gd8x\" (UID: \"27006609-ebbc-4519-a04e-0785af6dadbd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x" Jul 16 12:11:21.229977 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.229832 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.229977 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.229886 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-grpc-tls\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.229977 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.229910 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjr29\" (UniqueName: \"kubernetes.io/projected/27006609-ebbc-4519-a04e-0785af6dadbd-kube-api-access-gjr29\") pod \"monitoring-plugin-5f68df7f58-5gd8x\" (UID: \"27006609-ebbc-4519-a04e-0785af6dadbd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x" Jul 16 12:11:21.230747 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.230680 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5091f4e3-046e-4fe9-a173-976eb62cbd62-metrics-client-ca\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.233110 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.232942 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.233110 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.233027 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.233110 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.233036 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-grpc-tls\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.233110 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.233027 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.233387 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.233219 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-thanos-querier-tls\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.234583 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.234557 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5091f4e3-046e-4fe9-a173-976eb62cbd62-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.249103 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.249082 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4kvr\" (UniqueName: \"kubernetes.io/projected/5091f4e3-046e-4fe9-a173-976eb62cbd62-kube-api-access-d4kvr\") pod \"thanos-querier-58d688cdfc-wrpdp\" (UID: \"5091f4e3-046e-4fe9-a173-976eb62cbd62\") " pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.250855 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.250834 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:21.331357 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.331316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/27006609-ebbc-4519-a04e-0785af6dadbd-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-5gd8x\" (UID: \"27006609-ebbc-4519-a04e-0785af6dadbd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x" Jul 16 12:11:21.331839 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.331427 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjr29\" (UniqueName: \"kubernetes.io/projected/27006609-ebbc-4519-a04e-0785af6dadbd-kube-api-access-gjr29\") pod \"monitoring-plugin-5f68df7f58-5gd8x\" (UID: \"27006609-ebbc-4519-a04e-0785af6dadbd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x" Jul 16 12:11:21.331839 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:11:21.331495 2578 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 16 12:11:21.331839 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:11:21.331594 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27006609-ebbc-4519-a04e-0785af6dadbd-monitoring-plugin-cert podName:27006609-ebbc-4519-a04e-0785af6dadbd nodeName:}" failed. No retries permitted until 2026-07-16 12:11:21.831569604 +0000 UTC m=+231.594939053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/27006609-ebbc-4519-a04e-0785af6dadbd-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-5gd8x" (UID: "27006609-ebbc-4519-a04e-0785af6dadbd") : secret "monitoring-plugin-cert" not found Jul 16 12:11:21.344088 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.344065 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjr29\" (UniqueName: \"kubernetes.io/projected/27006609-ebbc-4519-a04e-0785af6dadbd-kube-api-access-gjr29\") pod \"monitoring-plugin-5f68df7f58-5gd8x\" (UID: \"27006609-ebbc-4519-a04e-0785af6dadbd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x" Jul 16 12:11:21.500877 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.500798 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-74c5dc9c58-wngvz"] Jul 16 12:11:21.503587 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.503565 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.508712 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.508656 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 16 12:11:21.508712 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.508675 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 16 12:11:21.508989 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.508973 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 16 12:11:21.509853 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.509827 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-b4fsx\"" Jul 16 12:11:21.509946 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.509870 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cf8sih43llffq\"" Jul 16 12:11:21.510144 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.510122 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 16 12:11:21.534032 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.533996 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/53c19468-fe34-48fb-b05a-513d02394a55-secret-metrics-server-client-certs\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.534218 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.534081 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6xn8\" (UniqueName: \"kubernetes.io/projected/53c19468-fe34-48fb-b05a-513d02394a55-kube-api-access-x6xn8\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.534288 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.534266 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/53c19468-fe34-48fb-b05a-513d02394a55-metrics-server-audit-profiles\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.534469 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.534448 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/53c19468-fe34-48fb-b05a-513d02394a55-audit-log\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.534572 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.534499 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53c19468-fe34-48fb-b05a-513d02394a55-client-ca-bundle\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.534629 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.534571 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53c19468-fe34-48fb-b05a-513d02394a55-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.534688 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.534626 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/53c19468-fe34-48fb-b05a-513d02394a55-secret-metrics-server-tls\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.542508 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.542481 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-74c5dc9c58-wngvz"] Jul 16 12:11:21.635610 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.635571 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53c19468-fe34-48fb-b05a-513d02394a55-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.635816 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.635624 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/53c19468-fe34-48fb-b05a-513d02394a55-secret-metrics-server-tls\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.635816 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.635665 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/53c19468-fe34-48fb-b05a-513d02394a55-secret-metrics-server-client-certs\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.635816 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.635703 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6xn8\" (UniqueName: \"kubernetes.io/projected/53c19468-fe34-48fb-b05a-513d02394a55-kube-api-access-x6xn8\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.635816 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.635784 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/53c19468-fe34-48fb-b05a-513d02394a55-metrics-server-audit-profiles\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.636023 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.635838 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/53c19468-fe34-48fb-b05a-513d02394a55-audit-log\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.636023 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.635879 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53c19468-fe34-48fb-b05a-513d02394a55-client-ca-bundle\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.636434 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.636392 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/53c19468-fe34-48fb-b05a-513d02394a55-audit-log\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.636552 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.636457 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53c19468-fe34-48fb-b05a-513d02394a55-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.636838 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.636788 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/53c19468-fe34-48fb-b05a-513d02394a55-metrics-server-audit-profiles\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.638842 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.638820 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/53c19468-fe34-48fb-b05a-513d02394a55-secret-metrics-server-tls\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.638943 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.638859 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53c19468-fe34-48fb-b05a-513d02394a55-client-ca-bundle\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.639290 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.639260 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/53c19468-fe34-48fb-b05a-513d02394a55-secret-metrics-server-client-certs\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.648178 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.648144 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6xn8\" (UniqueName: \"kubernetes.io/projected/53c19468-fe34-48fb-b05a-513d02394a55-kube-api-access-x6xn8\") pod \"metrics-server-74c5dc9c58-wngvz\" (UID: \"53c19468-fe34-48fb-b05a-513d02394a55\") " pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.752614 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.752528 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:21.814704 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.814673 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:21.838260 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.838216 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/27006609-ebbc-4519-a04e-0785af6dadbd-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-5gd8x\" (UID: \"27006609-ebbc-4519-a04e-0785af6dadbd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x" Jul 16 12:11:21.840966 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.840936 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/27006609-ebbc-4519-a04e-0785af6dadbd-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-5gd8x\" (UID: \"27006609-ebbc-4519-a04e-0785af6dadbd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x" Jul 16 12:11:21.990202 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:21.990167 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x" Jul 16 12:11:23.181176 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:23.181133 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:23.751299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:23.751244 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:23.871471 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:23.871429 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:26.202528 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:26.202460 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:26.329679 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:26.329619 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:26.881066 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:26.881025 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x"] Jul 16 12:11:26.911007 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:11:26.910971 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27006609_ebbc_4519_a04e_0785af6dadbd.slice/crio-4a3003f1b7db7ba1a526ae4fac28ee742da5b2eac7c3520c4047f6c7cf97e8a8 WatchSource:0}: Error finding container 4a3003f1b7db7ba1a526ae4fac28ee742da5b2eac7c3520c4047f6c7cf97e8a8: Status 404 returned error can't find the container with id 4a3003f1b7db7ba1a526ae4fac28ee742da5b2eac7c3520c4047f6c7cf97e8a8 Jul 16 12:11:27.093761 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:27.093677 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b8bc7b8cd-sdr5h"] Jul 16 12:11:27.097030 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:27.097001 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-74c5dc9c58-wngvz"] Jul 16 12:11:27.098127 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:11:27.098097 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod355b8eed_b03b_4f9f_8add_9163af2a965f.slice/crio-5d2ba91de11e70d4cb96b26467af0be6daf4650f0638f562bb6522aebe464fde WatchSource:0}: Error finding container 5d2ba91de11e70d4cb96b26467af0be6daf4650f0638f562bb6522aebe464fde: Status 404 returned error can't find the container with id 5d2ba91de11e70d4cb96b26467af0be6daf4650f0638f562bb6522aebe464fde Jul 16 12:11:27.101661 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:27.101436 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-58d688cdfc-wrpdp"] Jul 16 12:11:27.101661 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:11:27.101640 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c19468_fe34_48fb_b05a_513d02394a55.slice/crio-a7691f84dc541f0ab18a822f2c10773d2ea9baa07a2ee1d8631fd203af7bf25b WatchSource:0}: Error finding container a7691f84dc541f0ab18a822f2c10773d2ea9baa07a2ee1d8631fd203af7bf25b: Status 404 returned error can't find the container with id a7691f84dc541f0ab18a822f2c10773d2ea9baa07a2ee1d8631fd203af7bf25b Jul 16 12:11:27.107505 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:11:27.107471 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5091f4e3_046e_4fe9_a173_976eb62cbd62.slice/crio-5dd5a1c89cfc0371fe34e2862e5fb6ed9da2ad18a2e9427f2f5a6f3c2e89b28c WatchSource:0}: Error finding container 5dd5a1c89cfc0371fe34e2862e5fb6ed9da2ad18a2e9427f2f5a6f3c2e89b28c: Status 404 returned error can't find the container with id 5dd5a1c89cfc0371fe34e2862e5fb6ed9da2ad18a2e9427f2f5a6f3c2e89b28c Jul 16 12:11:27.293192 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:27.293153 2578 generic.go:356] "Generic (PLEG): container finished" podID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerID="b69f45b34dd77d95035a7b3888977160e9ad2a95c6a5e897f2e11050a695b460" exitCode=0 Jul 16 12:11:27.293689 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:27.293252 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28a3e2f1-66bb-41f2-afe1-79c2b8e67595","Type":"ContainerDied","Data":"b69f45b34dd77d95035a7b3888977160e9ad2a95c6a5e897f2e11050a695b460"} Jul 16 12:11:27.296454 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:27.296400 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nmpvs" event={"ID":"7601f52b-2d35-4c6c-a682-719c85050b81","Type":"ContainerStarted","Data":"4eae16c5423112918ff6c912d9d3b9acc23fb56bb9372b29ab4788b0e5275811"} Jul 16 12:11:27.296567 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:27.296487 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nmpvs" event={"ID":"7601f52b-2d35-4c6c-a682-719c85050b81","Type":"ContainerStarted","Data":"54528fe5f369a3dfe0c83ce74d942e6c3fb2f011fb7ced27d92fcf5a27f84338"} Jul 16 12:11:27.297802 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:27.297777 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" event={"ID":"5091f4e3-046e-4fe9-a173-976eb62cbd62","Type":"ContainerStarted","Data":"5dd5a1c89cfc0371fe34e2862e5fb6ed9da2ad18a2e9427f2f5a6f3c2e89b28c"} Jul 16 12:11:27.299622 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:27.299595 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-pvgch" event={"ID":"da0d38c7-23cc-4057-bb7a-0de38b2ee970","Type":"ContainerStarted","Data":"4e73087c18a674f8c56c31f6f05ba8eefd1c62518c2896583e4877de23514b3c"} Jul 16 12:11:27.299758 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:27.299741 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-pvgch" Jul 16 12:11:27.300965 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:27.300942 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x" event={"ID":"27006609-ebbc-4519-a04e-0785af6dadbd","Type":"ContainerStarted","Data":"4a3003f1b7db7ba1a526ae4fac28ee742da5b2eac7c3520c4047f6c7cf97e8a8"} Jul 16 12:11:27.302386 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:27.302361 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b8bc7b8cd-sdr5h" event={"ID":"355b8eed-b03b-4f9f-8add-9163af2a965f","Type":"ContainerStarted","Data":"5d2ba91de11e70d4cb96b26467af0be6daf4650f0638f562bb6522aebe464fde"} Jul 16 12:11:27.303543 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:27.303521 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" event={"ID":"53c19468-fe34-48fb-b05a-513d02394a55","Type":"ContainerStarted","Data":"a7691f84dc541f0ab18a822f2c10773d2ea9baa07a2ee1d8631fd203af7bf25b"} Jul 16 12:11:27.311811 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:27.311788 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-pvgch" Jul 16 12:11:27.357057 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:27.356995 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-pvgch" podStartSLOduration=1.255729484 podStartE2EDuration="19.356970449s" totalImagesPullingTime="18.101240965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:11:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:11:27.355726771 +0000 UTC m=+237.119096256" watchObservedRunningTime="2026-07-16 12:11:27.356970449 +0000 UTC m=+237.120339913" Jul 16 12:11:27.386492 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:27.386398 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nmpvs" podStartSLOduration=10.131617174 podStartE2EDuration="11.386375322s" totalImagesPullingTime="1.17909507s" totalInitContainerRuntime="75.663078ms" isStatefulPod=true podCreationTimestamp="2026-07-16 12:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:11:27.385708056 +0000 UTC m=+237.149077532" watchObservedRunningTime="2026-07-16 12:11:27.386375322 +0000 UTC m=+237.149744787" Jul 16 12:11:28.746238 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:28.745889 2578 patch_prober.go:28] interesting pod/image-registry-64d797f876-5qg7q container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 16 12:11:28.746238 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:28.745968 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-64d797f876-5qg7q" podUID="7331897a-45b7-4b04-be5c-9bb16e47b9a9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 12:11:28.854677 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:28.852501 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:28.950304 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:28.950176 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:29.563068 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:29.563023 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:29.671966 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:29.671924 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:30.215143 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:30.215108 2578 patch_prober.go:28] interesting pod/image-registry-64d797f876-5qg7q container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 16 12:11:30.215613 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:30.215183 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-64d797f876-5qg7q" podUID="7331897a-45b7-4b04-be5c-9bb16e47b9a9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 12:11:31.596792 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.596759 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7447fd5c77-qt5gr"] Jul 16 12:11:31.634558 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.634473 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7447fd5c77-qt5gr"] Jul 16 12:11:31.634723 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.634620 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.742924 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.742875 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79799e3b-f6d0-4a23-8e07-718e984d761b-console-serving-cert\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.742924 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.742924 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-trusted-ca-bundle\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.743166 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.742946 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-console-config\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.743166 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.743033 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-service-ca\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.743166 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.743070 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-oauth-serving-cert\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.743290 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.743247 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79799e3b-f6d0-4a23-8e07-718e984d761b-console-oauth-config\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.743290 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.743282 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vs58\" (UniqueName: \"kubernetes.io/projected/79799e3b-f6d0-4a23-8e07-718e984d761b-kube-api-access-4vs58\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.843826 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.843788 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79799e3b-f6d0-4a23-8e07-718e984d761b-console-oauth-config\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.844016 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.843834 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vs58\" (UniqueName: \"kubernetes.io/projected/79799e3b-f6d0-4a23-8e07-718e984d761b-kube-api-access-4vs58\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.844016 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.843869 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79799e3b-f6d0-4a23-8e07-718e984d761b-console-serving-cert\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.844016 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.843891 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-trusted-ca-bundle\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.844016 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.843917 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-console-config\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.844016 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.843952 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-service-ca\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.844016 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.843984 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-oauth-serving-cert\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.844830 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.844759 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-console-config\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.844968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.844827 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-service-ca\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.845378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.845142 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-trusted-ca-bundle\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.845378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.845277 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-oauth-serving-cert\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.846906 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.846841 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79799e3b-f6d0-4a23-8e07-718e984d761b-console-oauth-config\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.847070 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.847046 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79799e3b-f6d0-4a23-8e07-718e984d761b-console-serving-cert\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.856510 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.855880 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vs58\" (UniqueName: \"kubernetes.io/projected/79799e3b-f6d0-4a23-8e07-718e984d761b-kube-api-access-4vs58\") pod \"console-7447fd5c77-qt5gr\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:31.949831 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:31.949786 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:33.935797 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:33.931769 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7447fd5c77-qt5gr"] Jul 16 12:11:33.939138 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:11:33.939094 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79799e3b_f6d0_4a23_8e07_718e984d761b.slice/crio-0477edfddab3281635505c3b9f9a9d5379fdb24579568dfed42ef6b0fa129847 WatchSource:0}: Error finding container 0477edfddab3281635505c3b9f9a9d5379fdb24579568dfed42ef6b0fa129847: Status 404 returned error can't find the container with id 0477edfddab3281635505c3b9f9a9d5379fdb24579568dfed42ef6b0fa129847 Jul 16 12:11:34.345574 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.345522 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" event={"ID":"5091f4e3-046e-4fe9-a173-976eb62cbd62","Type":"ContainerStarted","Data":"47336c773ee35086ccc44f85371ce9f183312f823ee47d57afdcc75000560b48"} Jul 16 12:11:34.345681 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.345611 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" event={"ID":"5091f4e3-046e-4fe9-a173-976eb62cbd62","Type":"ContainerStarted","Data":"aad4f582ce7d172467a452f0ed3f5ea2d274e30d86e7bc6d411839acfab7caee"} Jul 16 12:11:34.345681 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.345659 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" event={"ID":"5091f4e3-046e-4fe9-a173-976eb62cbd62","Type":"ContainerStarted","Data":"92248a310d3859eb05cb5f283962341741654bdab118c7b9026b57dda3f78f25"} Jul 16 12:11:34.347581 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.347484 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x" event={"ID":"27006609-ebbc-4519-a04e-0785af6dadbd","Type":"ContainerStarted","Data":"9a2feb414fb475f58d69d2efa1c50c721684f2bee0a7adab4c7dc9a4d48563aa"} Jul 16 12:11:34.348696 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.348457 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x" Jul 16 12:11:34.350672 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.350534 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7447fd5c77-qt5gr" event={"ID":"79799e3b-f6d0-4a23-8e07-718e984d761b","Type":"ContainerStarted","Data":"e894828700f7a12b56688009f63b4045aa96958d4f8fd48368e449cf581ee689"} Jul 16 12:11:34.350672 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.350576 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7447fd5c77-qt5gr" event={"ID":"79799e3b-f6d0-4a23-8e07-718e984d761b","Type":"ContainerStarted","Data":"0477edfddab3281635505c3b9f9a9d5379fdb24579568dfed42ef6b0fa129847"} Jul 16 12:11:34.353784 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.353253 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b8bc7b8cd-sdr5h" event={"ID":"355b8eed-b03b-4f9f-8add-9163af2a965f","Type":"ContainerStarted","Data":"f2a4ea487dd52dd20f6d6c94fb16e8f5bf848dbcf804aa83f253ced1dacdb123"} Jul 16 12:11:34.354594 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.354537 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x" Jul 16 12:11:34.356033 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.355753 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" event={"ID":"53c19468-fe34-48fb-b05a-513d02394a55","Type":"ContainerStarted","Data":"8298c36f46953a481f2e21a0497841d63a5cefc20e879e8f416d28550b143843"} Jul 16 12:11:34.359798 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.359773 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28a3e2f1-66bb-41f2-afe1-79c2b8e67595","Type":"ContainerStarted","Data":"606cbd38be84b613aab8d9e6c842181e7885f73e7c1d6f4afdf2285f39cd3f32"} Jul 16 12:11:34.359913 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.359842 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28a3e2f1-66bb-41f2-afe1-79c2b8e67595","Type":"ContainerStarted","Data":"e3b3fc40ae5fb3b5746a6c9793d25d7257f842a8a915f85c86c6e8f5e4f39289"} Jul 16 12:11:34.359913 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.359886 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28a3e2f1-66bb-41f2-afe1-79c2b8e67595","Type":"ContainerStarted","Data":"59b61ac1956e5aecf13f22194d1504d80aa43281b0570919943224faec809350"} Jul 16 12:11:34.360030 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.359926 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28a3e2f1-66bb-41f2-afe1-79c2b8e67595","Type":"ContainerStarted","Data":"81f5c459c4cae4902075397bd864161c2f7906555ff9e5b8816116f2b4e46e65"} Jul 16 12:11:34.373382 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.373339 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-5gd8x" podStartSLOduration=6.559711741 podStartE2EDuration="13.37332391s" totalImagesPullingTime="6.813612169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:11:34.371340379 +0000 UTC m=+244.134709843" watchObservedRunningTime="2026-07-16 12:11:34.37332391 +0000 UTC m=+244.136693437" Jul 16 12:11:34.425812 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.425616 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" podStartSLOduration=6.802082971 podStartE2EDuration="13.425595415s" totalImagesPullingTime="6.623512444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:11:34.423262384 +0000 UTC m=+244.186631848" watchObservedRunningTime="2026-07-16 12:11:34.425595415 +0000 UTC m=+244.188964861" Jul 16 12:11:34.450743 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.450674 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b8bc7b8cd-sdr5h" podStartSLOduration=8.82125691 podStartE2EDuration="15.450650634s" totalImagesPullingTime="6.629393724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:11:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:11:34.445950152 +0000 UTC m=+244.209319617" watchObservedRunningTime="2026-07-16 12:11:34.450650634 +0000 UTC m=+244.214020097" Jul 16 12:11:34.471810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:34.471758 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7447fd5c77-qt5gr" podStartSLOduration=3.471738775 podStartE2EDuration="3.471738775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:11:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:11:34.469286551 +0000 UTC m=+244.232656015" watchObservedRunningTime="2026-07-16 12:11:34.471738775 +0000 UTC m=+244.235108237" Jul 16 12:11:35.369399 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:35.369360 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28a3e2f1-66bb-41f2-afe1-79c2b8e67595","Type":"ContainerStarted","Data":"2539a09df56e020f51b795f7f3e8b4d537873760697bb1f011cd23e2c6e4d937"} Jul 16 12:11:35.669080 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:35.669043 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:36.376697 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:36.376659 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28a3e2f1-66bb-41f2-afe1-79c2b8e67595","Type":"ContainerStarted","Data":"f9c81a65e9b9ee8b423228d6b1c1137354788297a7ddfe50cfc9b0de43f913d2"} Jul 16 12:11:36.379642 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:36.379617 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" event={"ID":"5091f4e3-046e-4fe9-a173-976eb62cbd62","Type":"ContainerStarted","Data":"60d6f7e73d399e4d839b90ca894d79769b7aad35eed7e212e343e850a6af0226"} Jul 16 12:11:36.379754 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:36.379677 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" event={"ID":"5091f4e3-046e-4fe9-a173-976eb62cbd62","Type":"ContainerStarted","Data":"e9a365863b27a6a9c22cae92dc1a4ea19fa5653abace3128f44b7e59bd5411ac"} Jul 16 12:11:36.379754 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:36.379714 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" event={"ID":"5091f4e3-046e-4fe9-a173-976eb62cbd62","Type":"ContainerStarted","Data":"d8de0d96d9d76cf069c4fb34c0083875955ff63db4a1a30c0d97570b6ffb0291"} Jul 16 12:11:36.410725 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:36.410683 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.436499987 podStartE2EDuration="19.410664401s" totalImagesPullingTime="15.950975572s" totalInitContainerRuntime="23.188842ms" isStatefulPod=false podCreationTimestamp="2026-07-16 12:11:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:11:36.409539901 +0000 UTC m=+246.172909365" watchObservedRunningTime="2026-07-16 12:11:36.410664401 +0000 UTC m=+246.174033866" Jul 16 12:11:36.444866 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:36.444813 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" podStartSLOduration=8.278879931 podStartE2EDuration="16.444792164s" totalImagesPullingTime="8.165912233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:11:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:11:36.441397806 +0000 UTC m=+246.204767270" watchObservedRunningTime="2026-07-16 12:11:36.444792164 +0000 UTC m=+246.208161628" Jul 16 12:11:37.384256 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:37.384226 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:38.741945 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:38.741911 2578 patch_prober.go:28] interesting pod/image-registry-64d797f876-5qg7q container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 16 12:11:38.742339 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:38.741969 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-64d797f876-5qg7q" podUID="7331897a-45b7-4b04-be5c-9bb16e47b9a9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 12:11:40.075387 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:40.075343 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:40.075885 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:40.075464 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:40.076714 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:40.076688 2578 patch_prober.go:28] interesting pod/console-7b8bc7b8cd-sdr5h container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.20:8443/health\": dial tcp 10.134.0.20:8443: connect: connection refused" start-of-body= Jul 16 12:11:40.076805 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:40.076749 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7b8bc7b8cd-sdr5h" podUID="355b8eed-b03b-4f9f-8add-9163af2a965f" containerName="console" probeResult="failure" output="Get \"https://10.134.0.20:8443/health\": dial tcp 10.134.0.20:8443: connect: connection refused" Jul 16 12:11:40.215164 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:40.215130 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64d797f876-5qg7q" Jul 16 12:11:40.456128 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:40.456043 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:40.574800 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:40.574767 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:40.654499 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:40.654459 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:41.815376 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:41.815336 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:41.815812 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:41.815408 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:11:41.950173 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:41.950134 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:41.950332 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:41.950200 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:41.951768 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:41.951748 2578 patch_prober.go:28] interesting pod/console-7447fd5c77-qt5gr container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.24:8443/health\": dial tcp 10.134.0.24:8443: connect: connection refused" start-of-body= Jul 16 12:11:41.951830 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:41.951812 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7447fd5c77-qt5gr" podUID="79799e3b-f6d0-4a23-8e07-718e984d761b" containerName="console" probeResult="failure" output="Get \"https://10.134.0.24:8443/health\": dial tcp 10.134.0.24:8443: connect: connection refused" Jul 16 12:11:42.460508 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:42.460468 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs\") pod \"network-metrics-daemon-rdpjz\" (UID: \"e73a1e88-85a5-4b47-bf8f-372eec90eafd\") " pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:11:42.462739 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:42.462716 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e73a1e88-85a5-4b47-bf8f-372eec90eafd-metrics-certs\") pod \"network-metrics-daemon-rdpjz\" (UID: \"e73a1e88-85a5-4b47-bf8f-372eec90eafd\") " pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:11:42.477562 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:42.477536 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w29rm\"" Jul 16 12:11:42.484931 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:42.484913 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdpjz" Jul 16 12:11:42.715270 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:42.715187 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rdpjz"] Jul 16 12:11:42.718425 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:11:42.718377 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode73a1e88_85a5_4b47_bf8f_372eec90eafd.slice/crio-7ccbfcd61c820051140f466022b1b6446921b07d5c9adefafe9eb8576d5a3b4e WatchSource:0}: Error finding container 7ccbfcd61c820051140f466022b1b6446921b07d5c9adefafe9eb8576d5a3b4e: Status 404 returned error can't find the container with id 7ccbfcd61c820051140f466022b1b6446921b07d5c9adefafe9eb8576d5a3b4e Jul 16 12:11:43.394786 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:43.394753 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-58d688cdfc-wrpdp" Jul 16 12:11:43.407890 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:43.407852 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rdpjz" event={"ID":"e73a1e88-85a5-4b47-bf8f-372eec90eafd","Type":"ContainerStarted","Data":"7ccbfcd61c820051140f466022b1b6446921b07d5c9adefafe9eb8576d5a3b4e"} Jul 16 12:11:43.849967 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:43.849926 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:43.950200 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:43.950169 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:44.415042 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:44.414942 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rdpjz" event={"ID":"e73a1e88-85a5-4b47-bf8f-372eec90eafd","Type":"ContainerStarted","Data":"728d9ef46cf3532f5a6b45ef35303d597112e80f172d5de9d89b1475ebd203c7"} Jul 16 12:11:44.415042 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:44.414996 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rdpjz" event={"ID":"e73a1e88-85a5-4b47-bf8f-372eec90eafd","Type":"ContainerStarted","Data":"a9beecb08b2a086cdfef3dbd20d23ee9adfc82607321970f7a6ad9f801a3ec6b"} Jul 16 12:11:44.445911 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:44.445850 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rdpjz" podStartSLOduration=251.121531313 podStartE2EDuration="4m12.445831596s" totalImagesPullingTime="1.324300283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:11:44.445746618 +0000 UTC m=+254.209116081" watchObservedRunningTime="2026-07-16 12:11:44.445831596 +0000 UTC m=+254.209201059" Jul 16 12:11:44.666815 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:44.666726 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:44.759515 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:44.759468 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:49.024189 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:49.024108 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:49.125786 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:49.125748 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:50.080052 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:50.080021 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:50.084607 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:50.084580 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:11:50.351651 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:50.351556 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:50.440744 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:50.440693 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:51.954260 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:51.954234 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:51.957933 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:51.957909 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:11:52.048857 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:52.048821 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b8bc7b8cd-sdr5h"] Jul 16 12:11:52.186073 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:52.186040 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:52.289088 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:52.289051 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:53.131818 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:53.131779 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:53.198727 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:53.198691 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:58.827575 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:58.827534 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:11:58.943379 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:11:58.943348 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:01.080560 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:01.080514 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:01.159933 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:01.159900 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:01.820440 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:01.820389 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:12:01.824366 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:01.824344 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-74c5dc9c58-wngvz" Jul 16 12:12:04.505344 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:04.505306 2578 generic.go:356] "Generic (PLEG): container finished" podID="fdb05941-6cba-40f2-8150-7465f6b82a78" containerID="0d7d7ef7789652fd672a71da00df97c408a2b9a2d1e6e07eb6d7bef913d8fc12" exitCode=0 Jul 16 12:12:04.505722 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:04.505384 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd" event={"ID":"fdb05941-6cba-40f2-8150-7465f6b82a78","Type":"ContainerDied","Data":"0d7d7ef7789652fd672a71da00df97c408a2b9a2d1e6e07eb6d7bef913d8fc12"} Jul 16 12:12:04.505812 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:04.505793 2578 scope.go:117] "RemoveContainer" containerID="0d7d7ef7789652fd672a71da00df97c408a2b9a2d1e6e07eb6d7bef913d8fc12" Jul 16 12:12:05.510847 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:05.510811 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rcrbd" event={"ID":"fdb05941-6cba-40f2-8150-7465f6b82a78","Type":"ContainerStarted","Data":"7191ee315b9cc08cee2a9898405aa2fa6c8f15808195f67b18426757504b25b4"} Jul 16 12:12:08.522978 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:08.522942 2578 generic.go:356] "Generic (PLEG): container finished" podID="f73a8a43-dd26-4e85-865c-a1aba38315c0" containerID="0e492905675c6663e71bf0c7e2db5fa34f35752da90ce7ef99a39e38ad6e991b" exitCode=0 Jul 16 12:12:08.523364 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:08.523017 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-277mm" event={"ID":"f73a8a43-dd26-4e85-865c-a1aba38315c0","Type":"ContainerDied","Data":"0e492905675c6663e71bf0c7e2db5fa34f35752da90ce7ef99a39e38ad6e991b"} Jul 16 12:12:08.523364 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:08.523341 2578 scope.go:117] "RemoveContainer" containerID="0e492905675c6663e71bf0c7e2db5fa34f35752da90ce7ef99a39e38ad6e991b" Jul 16 12:12:09.528703 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:09.528665 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-277mm" event={"ID":"f73a8a43-dd26-4e85-865c-a1aba38315c0","Type":"ContainerStarted","Data":"39ca896a4d968261a383f6f2ca4e2c763e0628d123c59eb62606c0470a8d304f"} Jul 16 12:12:09.824348 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:09.824311 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:09.945228 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:09.945186 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:10.292011 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:10.291973 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:10.430052 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:10.430021 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:11.270403 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:11.270372 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r7c42_e00ecaab-22f9-4338-ac72-f0eaad408347/dns-node-resolver/0.log" Jul 16 12:12:11.467132 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:11.467105 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68956567b4-ccm7k_060e6722-ce48-468c-aa89-cc07918db901/router/0.log" Jul 16 12:12:11.660826 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:11.660793 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-knfq6_10ea27e0-362a-4adb-a93c-e3db335e4610/serve-healthcheck-canary/0.log" Jul 16 12:12:13.552145 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:13.552108 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:13.640718 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:13.640683 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:14.010406 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:12:14.010315 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qblpk" podUID="8174e182-bed0-49f1-bdae-aeaaa73fd83c" Jul 16 12:12:14.548798 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:14.548766 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qblpk" Jul 16 12:12:17.072660 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.072594 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b8bc7b8cd-sdr5h" podUID="355b8eed-b03b-4f9f-8add-9163af2a965f" containerName="console" containerID="cri-o://f2a4ea487dd52dd20f6d6c94fb16e8f5bf848dbcf804aa83f253ced1dacdb123" gracePeriod=15 Jul 16 12:12:17.173825 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.173791 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:12:17.176227 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.176198 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8174e182-bed0-49f1-bdae-aeaaa73fd83c-metrics-tls\") pod \"dns-default-qblpk\" (UID: \"8174e182-bed0-49f1-bdae-aeaaa73fd83c\") " pod="openshift-dns/dns-default-qblpk" Jul 16 12:12:17.256755 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.256722 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v8966\"" Jul 16 12:12:17.260638 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.260607 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qblpk" Jul 16 12:12:17.360241 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.360213 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:12:17.426259 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.426227 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qblpk"] Jul 16 12:12:17.429857 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:12:17.429834 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8174e182_bed0_49f1_bdae_aeaaa73fd83c.slice/crio-30fca8c40b49da010222f950d226f238a544e1032a7b39cf9bd990d902338acb WatchSource:0}: Error finding container 30fca8c40b49da010222f950d226f238a544e1032a7b39cf9bd990d902338acb: Status 404 returned error can't find the container with id 30fca8c40b49da010222f950d226f238a544e1032a7b39cf9bd990d902338acb Jul 16 12:12:17.475770 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.475742 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/355b8eed-b03b-4f9f-8add-9163af2a965f-console-serving-cert\") pod \"355b8eed-b03b-4f9f-8add-9163af2a965f\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " Jul 16 12:12:17.475932 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.475792 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqf5x\" (UniqueName: \"kubernetes.io/projected/355b8eed-b03b-4f9f-8add-9163af2a965f-kube-api-access-lqf5x\") pod \"355b8eed-b03b-4f9f-8add-9163af2a965f\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " Jul 16 12:12:17.475932 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.475810 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-service-ca\") pod \"355b8eed-b03b-4f9f-8add-9163af2a965f\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " Jul 16 12:12:17.475932 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.475838 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-trusted-ca-bundle\") pod \"355b8eed-b03b-4f9f-8add-9163af2a965f\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " Jul 16 12:12:17.475932 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.475896 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-console-config\") pod \"355b8eed-b03b-4f9f-8add-9163af2a965f\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " Jul 16 12:12:17.475932 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.475932 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-oauth-serving-cert\") pod \"355b8eed-b03b-4f9f-8add-9163af2a965f\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " Jul 16 12:12:17.476206 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.475963 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/355b8eed-b03b-4f9f-8add-9163af2a965f-console-oauth-config\") pod \"355b8eed-b03b-4f9f-8add-9163af2a965f\" (UID: \"355b8eed-b03b-4f9f-8add-9163af2a965f\") " Jul 16 12:12:17.476399 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.476373 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-service-ca" (OuterVolumeSpecName: "service-ca") pod "355b8eed-b03b-4f9f-8add-9163af2a965f" (UID: "355b8eed-b03b-4f9f-8add-9163af2a965f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:12:17.476573 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.476502 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-console-config" (OuterVolumeSpecName: "console-config") pod "355b8eed-b03b-4f9f-8add-9163af2a965f" (UID: "355b8eed-b03b-4f9f-8add-9163af2a965f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:12:17.476573 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.476538 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "355b8eed-b03b-4f9f-8add-9163af2a965f" (UID: "355b8eed-b03b-4f9f-8add-9163af2a965f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:12:17.476738 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.476611 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "355b8eed-b03b-4f9f-8add-9163af2a965f" (UID: "355b8eed-b03b-4f9f-8add-9163af2a965f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:12:17.478658 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.478632 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/355b8eed-b03b-4f9f-8add-9163af2a965f-kube-api-access-lqf5x" (OuterVolumeSpecName: "kube-api-access-lqf5x") pod "355b8eed-b03b-4f9f-8add-9163af2a965f" (UID: "355b8eed-b03b-4f9f-8add-9163af2a965f"). InnerVolumeSpecName "kube-api-access-lqf5x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:12:17.478764 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.478658 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/355b8eed-b03b-4f9f-8add-9163af2a965f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "355b8eed-b03b-4f9f-8add-9163af2a965f" (UID: "355b8eed-b03b-4f9f-8add-9163af2a965f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:12:17.478822 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.478806 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/355b8eed-b03b-4f9f-8add-9163af2a965f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "355b8eed-b03b-4f9f-8add-9163af2a965f" (UID: "355b8eed-b03b-4f9f-8add-9163af2a965f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:12:17.562230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.562196 2578 generic.go:356] "Generic (PLEG): container finished" podID="355b8eed-b03b-4f9f-8add-9163af2a965f" containerID="f2a4ea487dd52dd20f6d6c94fb16e8f5bf848dbcf804aa83f253ced1dacdb123" exitCode=0 Jul 16 12:12:17.562435 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.562242 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b8bc7b8cd-sdr5h" Jul 16 12:12:17.562435 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.562286 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b8bc7b8cd-sdr5h" event={"ID":"355b8eed-b03b-4f9f-8add-9163af2a965f","Type":"ContainerDied","Data":"f2a4ea487dd52dd20f6d6c94fb16e8f5bf848dbcf804aa83f253ced1dacdb123"} Jul 16 12:12:17.562435 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.562339 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b8bc7b8cd-sdr5h" event={"ID":"355b8eed-b03b-4f9f-8add-9163af2a965f","Type":"ContainerDied","Data":"5d2ba91de11e70d4cb96b26467af0be6daf4650f0638f562bb6522aebe464fde"} Jul 16 12:12:17.562435 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.562366 2578 scope.go:117] "RemoveContainer" containerID="f2a4ea487dd52dd20f6d6c94fb16e8f5bf848dbcf804aa83f253ced1dacdb123" Jul 16 12:12:17.563721 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.563696 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qblpk" event={"ID":"8174e182-bed0-49f1-bdae-aeaaa73fd83c","Type":"ContainerStarted","Data":"30fca8c40b49da010222f950d226f238a544e1032a7b39cf9bd990d902338acb"} Jul 16 12:12:17.570378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.570359 2578 scope.go:117] "RemoveContainer" containerID="f2a4ea487dd52dd20f6d6c94fb16e8f5bf848dbcf804aa83f253ced1dacdb123" Jul 16 12:12:17.570666 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:12:17.570646 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2a4ea487dd52dd20f6d6c94fb16e8f5bf848dbcf804aa83f253ced1dacdb123\": container with ID starting with f2a4ea487dd52dd20f6d6c94fb16e8f5bf848dbcf804aa83f253ced1dacdb123 not found: ID does not exist" containerID="f2a4ea487dd52dd20f6d6c94fb16e8f5bf848dbcf804aa83f253ced1dacdb123" Jul 16 12:12:17.570718 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.570674 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a4ea487dd52dd20f6d6c94fb16e8f5bf848dbcf804aa83f253ced1dacdb123"} err="failed to get container status \"f2a4ea487dd52dd20f6d6c94fb16e8f5bf848dbcf804aa83f253ced1dacdb123\": rpc error: code = NotFound desc = could not find container \"f2a4ea487dd52dd20f6d6c94fb16e8f5bf848dbcf804aa83f253ced1dacdb123\": container with ID starting with f2a4ea487dd52dd20f6d6c94fb16e8f5bf848dbcf804aa83f253ced1dacdb123 not found: ID does not exist" Jul 16 12:12:17.577297 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.577245 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-trusted-ca-bundle\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:17.577297 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.577275 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-console-config\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:17.577297 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.577288 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-oauth-serving-cert\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:17.577444 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.577301 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/355b8eed-b03b-4f9f-8add-9163af2a965f-console-oauth-config\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:17.577444 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.577314 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/355b8eed-b03b-4f9f-8add-9163af2a965f-console-serving-cert\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:17.577444 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.577326 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqf5x\" (UniqueName: \"kubernetes.io/projected/355b8eed-b03b-4f9f-8add-9163af2a965f-kube-api-access-lqf5x\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:17.577444 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.577339 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/355b8eed-b03b-4f9f-8add-9163af2a965f-service-ca\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:17.588724 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.588689 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b8bc7b8cd-sdr5h"] Jul 16 12:12:17.593769 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:17.593746 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b8bc7b8cd-sdr5h"] Jul 16 12:12:19.478953 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:19.478917 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="355b8eed-b03b-4f9f-8add-9163af2a965f" path="/var/lib/kubelet/pods/355b8eed-b03b-4f9f-8add-9163af2a965f/volumes" Jul 16 12:12:20.580935 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:20.580899 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qblpk" event={"ID":"8174e182-bed0-49f1-bdae-aeaaa73fd83c","Type":"ContainerStarted","Data":"e350d2d1be29d843d55e1a2dfac25bf5465337505fc3554cd84cfb768a8446ea"} Jul 16 12:12:20.581304 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:20.580954 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qblpk" event={"ID":"8174e182-bed0-49f1-bdae-aeaaa73fd83c","Type":"ContainerStarted","Data":"4966e365d07e9584fff4a47bd2a54b23d36b8c344ea36a31865c7c654420b57f"} Jul 16 12:12:20.581304 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:20.581009 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qblpk" Jul 16 12:12:20.613113 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:20.613055 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qblpk" podStartSLOduration=251.541151482 podStartE2EDuration="4m13.613038453s" totalImagesPullingTime="2.071886971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:12:20.611757608 +0000 UTC m=+290.375127072" watchObservedRunningTime="2026-07-16 12:12:20.613038453 +0000 UTC m=+290.376407915" Jul 16 12:12:21.772523 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:21.772479 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:21.894357 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:21.894312 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:21.995231 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:21.995193 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:28.815830 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:28.815787 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:28.894978 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:28.894940 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:30.587040 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:30.587014 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qblpk" Jul 16 12:12:30.627593 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:30.627564 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:12:30.632599 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:30.632548 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:12:30.635859 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:30.635826 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:12:30.640879 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:30.640862 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 12:12:30.641274 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:30.641259 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:12:32.974878 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:32.974838 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:33.077542 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:33.077501 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:37.295290 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.295253 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 12:12:37.295769 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.295567 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="alertmanager" containerID="cri-o://81f5c459c4cae4902075397bd864161c2f7906555ff9e5b8816116f2b4e46e65" gracePeriod=120 Jul 16 12:12:37.295769 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.295613 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="kube-rbac-proxy-metric" containerID="cri-o://2539a09df56e020f51b795f7f3e8b4d537873760697bb1f011cd23e2c6e4d937" gracePeriod=120 Jul 16 12:12:37.295769 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.295653 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="kube-rbac-proxy-web" containerID="cri-o://e3b3fc40ae5fb3b5746a6c9793d25d7257f842a8a915f85c86c6e8f5e4f39289" gracePeriod=120 Jul 16 12:12:37.295769 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.295706 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="prom-label-proxy" containerID="cri-o://f9c81a65e9b9ee8b423228d6b1c1137354788297a7ddfe50cfc9b0de43f913d2" gracePeriod=120 Jul 16 12:12:37.295769 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.295729 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="kube-rbac-proxy" containerID="cri-o://606cbd38be84b613aab8d9e6c842181e7885f73e7c1d6f4afdf2285f39cd3f32" gracePeriod=120 Jul 16 12:12:37.295769 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.295655 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="config-reloader" containerID="cri-o://59b61ac1956e5aecf13f22194d1504d80aa43281b0570919943224faec809350" gracePeriod=120 Jul 16 12:12:37.380372 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.380315 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9df88b6cb-s5p8q"] Jul 16 12:12:37.380791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.380771 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="355b8eed-b03b-4f9f-8add-9163af2a965f" containerName="console" Jul 16 12:12:37.380791 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.380792 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="355b8eed-b03b-4f9f-8add-9163af2a965f" containerName="console" Jul 16 12:12:37.381011 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.380892 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="355b8eed-b03b-4f9f-8add-9163af2a965f" containerName="console" Jul 16 12:12:37.383642 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.383609 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.398846 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.398819 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9df88b6cb-s5p8q"] Jul 16 12:12:37.479203 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.479169 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:37.558007 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.557915 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-console-config\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.558007 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.557963 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-oauth-serving-cert\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.558007 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.557993 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-trusted-ca-bundle\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.558209 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.558027 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab04ad52-6514-473e-9852-6767cbf5291b-console-serving-cert\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.558209 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.558078 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab04ad52-6514-473e-9852-6767cbf5291b-console-oauth-config\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.558209 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.558119 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-service-ca\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.558209 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.558138 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk9c4\" (UniqueName: \"kubernetes.io/projected/ab04ad52-6514-473e-9852-6767cbf5291b-kube-api-access-sk9c4\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.583099 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.583061 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:37.652030 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.651991 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:37.658527 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.658504 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab04ad52-6514-473e-9852-6767cbf5291b-console-oauth-config\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.658702 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.658537 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-service-ca\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.658702 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.658558 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk9c4\" (UniqueName: \"kubernetes.io/projected/ab04ad52-6514-473e-9852-6767cbf5291b-kube-api-access-sk9c4\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.658702 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.658668 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-console-config\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.658865 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.658717 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-oauth-serving-cert\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.658865 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.658766 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-trusted-ca-bundle\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.658865 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.658803 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab04ad52-6514-473e-9852-6767cbf5291b-console-serving-cert\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.659346 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.659324 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-service-ca\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.659537 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.659374 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-console-config\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.659537 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.659518 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-trusted-ca-bundle\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.659698 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.659565 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-oauth-serving-cert\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.661514 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.661481 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab04ad52-6514-473e-9852-6767cbf5291b-console-oauth-config\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.661637 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.661618 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab04ad52-6514-473e-9852-6767cbf5291b-console-serving-cert\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.662043 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.662023 2578 generic.go:356] "Generic (PLEG): container finished" podID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerID="f9c81a65e9b9ee8b423228d6b1c1137354788297a7ddfe50cfc9b0de43f913d2" exitCode=0 Jul 16 12:12:37.662043 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.662042 2578 generic.go:356] "Generic (PLEG): container finished" podID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerID="2539a09df56e020f51b795f7f3e8b4d537873760697bb1f011cd23e2c6e4d937" exitCode=0 Jul 16 12:12:37.662156 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.662047 2578 generic.go:356] "Generic (PLEG): container finished" podID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerID="606cbd38be84b613aab8d9e6c842181e7885f73e7c1d6f4afdf2285f39cd3f32" exitCode=0 Jul 16 12:12:37.662156 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.662053 2578 generic.go:356] "Generic (PLEG): container finished" podID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerID="59b61ac1956e5aecf13f22194d1504d80aa43281b0570919943224faec809350" exitCode=0 Jul 16 12:12:37.662156 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.662058 2578 generic.go:356] "Generic (PLEG): container finished" podID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerID="81f5c459c4cae4902075397bd864161c2f7906555ff9e5b8816116f2b4e46e65" exitCode=0 Jul 16 12:12:37.662156 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.662090 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28a3e2f1-66bb-41f2-afe1-79c2b8e67595","Type":"ContainerDied","Data":"f9c81a65e9b9ee8b423228d6b1c1137354788297a7ddfe50cfc9b0de43f913d2"} Jul 16 12:12:37.662156 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.662152 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28a3e2f1-66bb-41f2-afe1-79c2b8e67595","Type":"ContainerDied","Data":"2539a09df56e020f51b795f7f3e8b4d537873760697bb1f011cd23e2c6e4d937"} Jul 16 12:12:37.662307 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.662178 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28a3e2f1-66bb-41f2-afe1-79c2b8e67595","Type":"ContainerDied","Data":"606cbd38be84b613aab8d9e6c842181e7885f73e7c1d6f4afdf2285f39cd3f32"} Jul 16 12:12:37.662307 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.662202 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28a3e2f1-66bb-41f2-afe1-79c2b8e67595","Type":"ContainerDied","Data":"59b61ac1956e5aecf13f22194d1504d80aa43281b0570919943224faec809350"} Jul 16 12:12:37.662307 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.662226 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28a3e2f1-66bb-41f2-afe1-79c2b8e67595","Type":"ContainerDied","Data":"81f5c459c4cae4902075397bd864161c2f7906555ff9e5b8816116f2b4e46e65"} Jul 16 12:12:37.667108 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.667089 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk9c4\" (UniqueName: \"kubernetes.io/projected/ab04ad52-6514-473e-9852-6767cbf5291b-kube-api-access-sk9c4\") pod \"console-9df88b6cb-s5p8q\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.699211 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.699179 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:37.840561 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:37.840541 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9df88b6cb-s5p8q"] Jul 16 12:12:37.843085 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:12:37.843054 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab04ad52_6514_473e_9852_6767cbf5291b.slice/crio-ccc2c6093960bd3a96e4beef0aad64d92f3101e98e0a21a675e4bcc65e40ee8a WatchSource:0}: Error finding container ccc2c6093960bd3a96e4beef0aad64d92f3101e98e0a21a675e4bcc65e40ee8a: Status 404 returned error can't find the container with id ccc2c6093960bd3a96e4beef0aad64d92f3101e98e0a21a675e4bcc65e40ee8a Jul 16 12:12:38.170309 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.170216 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:38.239122 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.239091 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:38.543849 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.543823 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:38.667805 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.667767 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-web-config\") pod \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " Jul 16 12:12:38.667805 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.667804 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-main-tls\") pod \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " Jul 16 12:12:38.668063 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.667764 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9df88b6cb-s5p8q" event={"ID":"ab04ad52-6514-473e-9852-6767cbf5291b","Type":"ContainerStarted","Data":"79226af62a80c54e242f090a3e82a911020287f1fb2b76ca5d93284b4a62a174"} Jul 16 12:12:38.668063 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.667837 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-config-out\") pod \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " Jul 16 12:12:38.668063 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.667862 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-tls-assets\") pod \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " Jul 16 12:12:38.668063 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.667863 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9df88b6cb-s5p8q" event={"ID":"ab04ad52-6514-473e-9852-6767cbf5291b","Type":"ContainerStarted","Data":"ccc2c6093960bd3a96e4beef0aad64d92f3101e98e0a21a675e4bcc65e40ee8a"} Jul 16 12:12:38.668063 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.667905 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-cluster-tls-config\") pod \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " Jul 16 12:12:38.668063 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.667947 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-alertmanager-main-db\") pod \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " Jul 16 12:12:38.668063 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.667980 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy-web\") pod \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " Jul 16 12:12:38.668063 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.668028 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmsvp\" (UniqueName: \"kubernetes.io/projected/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-kube-api-access-zmsvp\") pod \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " Jul 16 12:12:38.668063 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.668046 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy\") pod \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " Jul 16 12:12:38.668534 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.668079 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-alertmanager-trusted-ca-bundle\") pod \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " Jul 16 12:12:38.668534 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.668108 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-metrics-client-ca\") pod \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " Jul 16 12:12:38.668534 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.668137 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-config-volume\") pod \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " Jul 16 12:12:38.668534 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.668157 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy-metric\") pod \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\" (UID: \"28a3e2f1-66bb-41f2-afe1-79c2b8e67595\") " Jul 16 12:12:38.669110 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.668849 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "28a3e2f1-66bb-41f2-afe1-79c2b8e67595" (UID: "28a3e2f1-66bb-41f2-afe1-79c2b8e67595"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:12:38.669378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.669352 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "28a3e2f1-66bb-41f2-afe1-79c2b8e67595" (UID: "28a3e2f1-66bb-41f2-afe1-79c2b8e67595"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:12:38.669697 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.669672 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "28a3e2f1-66bb-41f2-afe1-79c2b8e67595" (UID: "28a3e2f1-66bb-41f2-afe1-79c2b8e67595"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:12:38.671169 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.671143 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "28a3e2f1-66bb-41f2-afe1-79c2b8e67595" (UID: "28a3e2f1-66bb-41f2-afe1-79c2b8e67595"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:12:38.672215 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.672100 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-config-volume" (OuterVolumeSpecName: "config-volume") pod "28a3e2f1-66bb-41f2-afe1-79c2b8e67595" (UID: "28a3e2f1-66bb-41f2-afe1-79c2b8e67595"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:12:38.672215 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.672125 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-config-out" (OuterVolumeSpecName: "config-out") pod "28a3e2f1-66bb-41f2-afe1-79c2b8e67595" (UID: "28a3e2f1-66bb-41f2-afe1-79c2b8e67595"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:12:38.672215 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.672176 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-kube-api-access-zmsvp" (OuterVolumeSpecName: "kube-api-access-zmsvp") pod "28a3e2f1-66bb-41f2-afe1-79c2b8e67595" (UID: "28a3e2f1-66bb-41f2-afe1-79c2b8e67595"). InnerVolumeSpecName "kube-api-access-zmsvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:12:38.672401 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.672236 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "28a3e2f1-66bb-41f2-afe1-79c2b8e67595" (UID: "28a3e2f1-66bb-41f2-afe1-79c2b8e67595"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:12:38.672401 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.672362 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "28a3e2f1-66bb-41f2-afe1-79c2b8e67595" (UID: "28a3e2f1-66bb-41f2-afe1-79c2b8e67595"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:12:38.672764 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.672722 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "28a3e2f1-66bb-41f2-afe1-79c2b8e67595" (UID: "28a3e2f1-66bb-41f2-afe1-79c2b8e67595"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:12:38.673032 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.673010 2578 generic.go:356] "Generic (PLEG): container finished" podID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerID="e3b3fc40ae5fb3b5746a6c9793d25d7257f842a8a915f85c86c6e8f5e4f39289" exitCode=0 Jul 16 12:12:38.673127 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.673052 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28a3e2f1-66bb-41f2-afe1-79c2b8e67595","Type":"ContainerDied","Data":"e3b3fc40ae5fb3b5746a6c9793d25d7257f842a8a915f85c86c6e8f5e4f39289"} Jul 16 12:12:38.673127 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.673110 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:38.673234 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.673131 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28a3e2f1-66bb-41f2-afe1-79c2b8e67595","Type":"ContainerDied","Data":"44d84eee6e2646f2f90b8d3649627fa35589bbe5dc71db1aaad64949ad379dce"} Jul 16 12:12:38.673234 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.673183 2578 scope.go:117] "RemoveContainer" containerID="f9c81a65e9b9ee8b423228d6b1c1137354788297a7ddfe50cfc9b0de43f913d2" Jul 16 12:12:38.674008 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.673974 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "28a3e2f1-66bb-41f2-afe1-79c2b8e67595" (UID: "28a3e2f1-66bb-41f2-afe1-79c2b8e67595"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:12:38.677091 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.676990 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "28a3e2f1-66bb-41f2-afe1-79c2b8e67595" (UID: "28a3e2f1-66bb-41f2-afe1-79c2b8e67595"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:12:38.683788 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.683756 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-web-config" (OuterVolumeSpecName: "web-config") pod "28a3e2f1-66bb-41f2-afe1-79c2b8e67595" (UID: "28a3e2f1-66bb-41f2-afe1-79c2b8e67595"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:12:38.684889 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.684873 2578 scope.go:117] "RemoveContainer" containerID="2539a09df56e020f51b795f7f3e8b4d537873760697bb1f011cd23e2c6e4d937" Jul 16 12:12:38.692593 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.692278 2578 scope.go:117] "RemoveContainer" containerID="606cbd38be84b613aab8d9e6c842181e7885f73e7c1d6f4afdf2285f39cd3f32" Jul 16 12:12:38.693613 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.693575 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9df88b6cb-s5p8q" podStartSLOduration=1.693562171 podStartE2EDuration="1.693562171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:12:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:12:38.69096476 +0000 UTC m=+308.454334222" watchObservedRunningTime="2026-07-16 12:12:38.693562171 +0000 UTC m=+308.456931694" Jul 16 12:12:38.699793 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.699774 2578 scope.go:117] "RemoveContainer" containerID="e3b3fc40ae5fb3b5746a6c9793d25d7257f842a8a915f85c86c6e8f5e4f39289" Jul 16 12:12:38.706991 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.706971 2578 scope.go:117] "RemoveContainer" containerID="59b61ac1956e5aecf13f22194d1504d80aa43281b0570919943224faec809350" Jul 16 12:12:38.714232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.714217 2578 scope.go:117] "RemoveContainer" containerID="81f5c459c4cae4902075397bd864161c2f7906555ff9e5b8816116f2b4e46e65" Jul 16 12:12:38.721353 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.721335 2578 scope.go:117] "RemoveContainer" containerID="b69f45b34dd77d95035a7b3888977160e9ad2a95c6a5e897f2e11050a695b460" Jul 16 12:12:38.728195 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.728178 2578 scope.go:117] "RemoveContainer" containerID="f9c81a65e9b9ee8b423228d6b1c1137354788297a7ddfe50cfc9b0de43f913d2" Jul 16 12:12:38.728478 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:12:38.728454 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9c81a65e9b9ee8b423228d6b1c1137354788297a7ddfe50cfc9b0de43f913d2\": container with ID starting with f9c81a65e9b9ee8b423228d6b1c1137354788297a7ddfe50cfc9b0de43f913d2 not found: ID does not exist" containerID="f9c81a65e9b9ee8b423228d6b1c1137354788297a7ddfe50cfc9b0de43f913d2" Jul 16 12:12:38.728540 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.728488 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9c81a65e9b9ee8b423228d6b1c1137354788297a7ddfe50cfc9b0de43f913d2"} err="failed to get container status \"f9c81a65e9b9ee8b423228d6b1c1137354788297a7ddfe50cfc9b0de43f913d2\": rpc error: code = NotFound desc = could not find container \"f9c81a65e9b9ee8b423228d6b1c1137354788297a7ddfe50cfc9b0de43f913d2\": container with ID starting with f9c81a65e9b9ee8b423228d6b1c1137354788297a7ddfe50cfc9b0de43f913d2 not found: ID does not exist" Jul 16 12:12:38.728540 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.728513 2578 scope.go:117] "RemoveContainer" containerID="2539a09df56e020f51b795f7f3e8b4d537873760697bb1f011cd23e2c6e4d937" Jul 16 12:12:38.728735 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:12:38.728720 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2539a09df56e020f51b795f7f3e8b4d537873760697bb1f011cd23e2c6e4d937\": container with ID starting with 2539a09df56e020f51b795f7f3e8b4d537873760697bb1f011cd23e2c6e4d937 not found: ID does not exist" containerID="2539a09df56e020f51b795f7f3e8b4d537873760697bb1f011cd23e2c6e4d937" Jul 16 12:12:38.728770 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.728739 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2539a09df56e020f51b795f7f3e8b4d537873760697bb1f011cd23e2c6e4d937"} err="failed to get container status \"2539a09df56e020f51b795f7f3e8b4d537873760697bb1f011cd23e2c6e4d937\": rpc error: code = NotFound desc = could not find container \"2539a09df56e020f51b795f7f3e8b4d537873760697bb1f011cd23e2c6e4d937\": container with ID starting with 2539a09df56e020f51b795f7f3e8b4d537873760697bb1f011cd23e2c6e4d937 not found: ID does not exist" Jul 16 12:12:38.728770 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.728762 2578 scope.go:117] "RemoveContainer" containerID="606cbd38be84b613aab8d9e6c842181e7885f73e7c1d6f4afdf2285f39cd3f32" Jul 16 12:12:38.729004 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:12:38.728987 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"606cbd38be84b613aab8d9e6c842181e7885f73e7c1d6f4afdf2285f39cd3f32\": container with ID starting with 606cbd38be84b613aab8d9e6c842181e7885f73e7c1d6f4afdf2285f39cd3f32 not found: ID does not exist" containerID="606cbd38be84b613aab8d9e6c842181e7885f73e7c1d6f4afdf2285f39cd3f32" Jul 16 12:12:38.729053 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.729012 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"606cbd38be84b613aab8d9e6c842181e7885f73e7c1d6f4afdf2285f39cd3f32"} err="failed to get container status \"606cbd38be84b613aab8d9e6c842181e7885f73e7c1d6f4afdf2285f39cd3f32\": rpc error: code = NotFound desc = could not find container \"606cbd38be84b613aab8d9e6c842181e7885f73e7c1d6f4afdf2285f39cd3f32\": container with ID starting with 606cbd38be84b613aab8d9e6c842181e7885f73e7c1d6f4afdf2285f39cd3f32 not found: ID does not exist" Jul 16 12:12:38.729053 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.729028 2578 scope.go:117] "RemoveContainer" containerID="e3b3fc40ae5fb3b5746a6c9793d25d7257f842a8a915f85c86c6e8f5e4f39289" Jul 16 12:12:38.729267 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:12:38.729252 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3b3fc40ae5fb3b5746a6c9793d25d7257f842a8a915f85c86c6e8f5e4f39289\": container with ID starting with e3b3fc40ae5fb3b5746a6c9793d25d7257f842a8a915f85c86c6e8f5e4f39289 not found: ID does not exist" containerID="e3b3fc40ae5fb3b5746a6c9793d25d7257f842a8a915f85c86c6e8f5e4f39289" Jul 16 12:12:38.729305 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.729271 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3b3fc40ae5fb3b5746a6c9793d25d7257f842a8a915f85c86c6e8f5e4f39289"} err="failed to get container status \"e3b3fc40ae5fb3b5746a6c9793d25d7257f842a8a915f85c86c6e8f5e4f39289\": rpc error: code = NotFound desc = could not find container \"e3b3fc40ae5fb3b5746a6c9793d25d7257f842a8a915f85c86c6e8f5e4f39289\": container with ID starting with e3b3fc40ae5fb3b5746a6c9793d25d7257f842a8a915f85c86c6e8f5e4f39289 not found: ID does not exist" Jul 16 12:12:38.729305 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.729284 2578 scope.go:117] "RemoveContainer" containerID="59b61ac1956e5aecf13f22194d1504d80aa43281b0570919943224faec809350" Jul 16 12:12:38.729560 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:12:38.729542 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59b61ac1956e5aecf13f22194d1504d80aa43281b0570919943224faec809350\": container with ID starting with 59b61ac1956e5aecf13f22194d1504d80aa43281b0570919943224faec809350 not found: ID does not exist" containerID="59b61ac1956e5aecf13f22194d1504d80aa43281b0570919943224faec809350" Jul 16 12:12:38.729604 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.729571 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59b61ac1956e5aecf13f22194d1504d80aa43281b0570919943224faec809350"} err="failed to get container status \"59b61ac1956e5aecf13f22194d1504d80aa43281b0570919943224faec809350\": rpc error: code = NotFound desc = could not find container \"59b61ac1956e5aecf13f22194d1504d80aa43281b0570919943224faec809350\": container with ID starting with 59b61ac1956e5aecf13f22194d1504d80aa43281b0570919943224faec809350 not found: ID does not exist" Jul 16 12:12:38.729604 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.729587 2578 scope.go:117] "RemoveContainer" containerID="81f5c459c4cae4902075397bd864161c2f7906555ff9e5b8816116f2b4e46e65" Jul 16 12:12:38.729822 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:12:38.729806 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81f5c459c4cae4902075397bd864161c2f7906555ff9e5b8816116f2b4e46e65\": container with ID starting with 81f5c459c4cae4902075397bd864161c2f7906555ff9e5b8816116f2b4e46e65 not found: ID does not exist" containerID="81f5c459c4cae4902075397bd864161c2f7906555ff9e5b8816116f2b4e46e65" Jul 16 12:12:38.729872 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.729826 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81f5c459c4cae4902075397bd864161c2f7906555ff9e5b8816116f2b4e46e65"} err="failed to get container status \"81f5c459c4cae4902075397bd864161c2f7906555ff9e5b8816116f2b4e46e65\": rpc error: code = NotFound desc = could not find container \"81f5c459c4cae4902075397bd864161c2f7906555ff9e5b8816116f2b4e46e65\": container with ID starting with 81f5c459c4cae4902075397bd864161c2f7906555ff9e5b8816116f2b4e46e65 not found: ID does not exist" Jul 16 12:12:38.729872 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.729839 2578 scope.go:117] "RemoveContainer" containerID="b69f45b34dd77d95035a7b3888977160e9ad2a95c6a5e897f2e11050a695b460" Jul 16 12:12:38.730063 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:12:38.730047 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b69f45b34dd77d95035a7b3888977160e9ad2a95c6a5e897f2e11050a695b460\": container with ID starting with b69f45b34dd77d95035a7b3888977160e9ad2a95c6a5e897f2e11050a695b460 not found: ID does not exist" containerID="b69f45b34dd77d95035a7b3888977160e9ad2a95c6a5e897f2e11050a695b460" Jul 16 12:12:38.730105 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.730068 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b69f45b34dd77d95035a7b3888977160e9ad2a95c6a5e897f2e11050a695b460"} err="failed to get container status \"b69f45b34dd77d95035a7b3888977160e9ad2a95c6a5e897f2e11050a695b460\": rpc error: code = NotFound desc = could not find container \"b69f45b34dd77d95035a7b3888977160e9ad2a95c6a5e897f2e11050a695b460\": container with ID starting with b69f45b34dd77d95035a7b3888977160e9ad2a95c6a5e897f2e11050a695b460 not found: ID does not exist" Jul 16 12:12:38.769610 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.769502 2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:38.769610 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.769541 2578 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-metrics-client-ca\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:38.769610 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.769557 2578 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-config-volume\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:38.769610 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.769572 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:38.769610 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.769609 2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-web-config\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:38.769906 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.769633 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-main-tls\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:38.769906 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.769650 2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-config-out\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:38.769906 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.769663 2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-tls-assets\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:38.769906 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.769675 2578 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-cluster-tls-config\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:38.769906 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.769689 2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-alertmanager-main-db\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:38.769906 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.769702 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:38.769906 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.769715 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmsvp\" (UniqueName: \"kubernetes.io/projected/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-kube-api-access-zmsvp\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:38.769906 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:38.769727 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28a3e2f1-66bb-41f2-afe1-79c2b8e67595-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:12:39.004901 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.004859 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 12:12:39.006552 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.006529 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 12:12:39.051520 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051480 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 12:12:39.051827 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051807 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="kube-rbac-proxy-web" Jul 16 12:12:39.051827 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051823 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="kube-rbac-proxy-web" Jul 16 12:12:39.051987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051840 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="init-config-reloader" Jul 16 12:12:39.051987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051849 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="init-config-reloader" Jul 16 12:12:39.051987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051862 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="config-reloader" Jul 16 12:12:39.051987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051868 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="config-reloader" Jul 16 12:12:39.051987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051876 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="kube-rbac-proxy" Jul 16 12:12:39.051987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051881 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="kube-rbac-proxy" Jul 16 12:12:39.051987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051890 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="prom-label-proxy" Jul 16 12:12:39.051987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051895 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="prom-label-proxy" Jul 16 12:12:39.051987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051903 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="alertmanager" Jul 16 12:12:39.051987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051908 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="alertmanager" Jul 16 12:12:39.051987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051917 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="kube-rbac-proxy-metric" Jul 16 12:12:39.051987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051926 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="kube-rbac-proxy-metric" Jul 16 12:12:39.051987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051979 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="alertmanager" Jul 16 12:12:39.051987 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051988 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="kube-rbac-proxy" Jul 16 12:12:39.052488 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.051995 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="kube-rbac-proxy-metric" Jul 16 12:12:39.052488 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.052005 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="kube-rbac-proxy-web" Jul 16 12:12:39.052488 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.052014 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="config-reloader" Jul 16 12:12:39.052488 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.052020 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" containerName="prom-label-proxy" Jul 16 12:12:39.055856 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.055831 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.058994 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.058783 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 16 12:12:39.059609 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.059358 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 16 12:12:39.060095 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.059946 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 16 12:12:39.061036 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.060351 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 16 12:12:39.061036 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.060580 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 16 12:12:39.061161 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.061083 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 16 12:12:39.061161 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.061127 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 16 12:12:39.061259 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.061085 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 16 12:12:39.061318 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.061278 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-p2pjx\"" Jul 16 12:12:39.065130 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.065114 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 16 12:12:39.084072 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.084045 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 12:12:39.173975 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.173932 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.174176 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.173998 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.174176 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.174045 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/491953d0-3e2f-4b08-aacf-316d9631c006-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.174176 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.174128 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/491953d0-3e2f-4b08-aacf-316d9631c006-tls-assets\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.174176 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.174156 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-web-config\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.174176 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.174179 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.174451 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.174199 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491953d0-3e2f-4b08-aacf-316d9631c006-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.174451 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.174232 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-config-volume\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.174451 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.174251 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.174451 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.174267 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plz67\" (UniqueName: \"kubernetes.io/projected/491953d0-3e2f-4b08-aacf-316d9631c006-kube-api-access-plz67\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.174451 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.174289 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/491953d0-3e2f-4b08-aacf-316d9631c006-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.174451 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.174346 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/491953d0-3e2f-4b08-aacf-316d9631c006-config-out\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.174451 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.174366 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.275856 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.275762 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/491953d0-3e2f-4b08-aacf-316d9631c006-config-out\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.275856 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.275804 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.275856 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.275841 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.276121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.275885 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.276121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.275920 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/491953d0-3e2f-4b08-aacf-316d9631c006-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.276121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.275971 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/491953d0-3e2f-4b08-aacf-316d9631c006-tls-assets\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.276121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.275987 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-web-config\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.276121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.276012 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.276121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.276041 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491953d0-3e2f-4b08-aacf-316d9631c006-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.276121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.276064 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-config-volume\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.276121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.276086 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.276121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.276109 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plz67\" (UniqueName: \"kubernetes.io/projected/491953d0-3e2f-4b08-aacf-316d9631c006-kube-api-access-plz67\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.276600 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.276131 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/491953d0-3e2f-4b08-aacf-316d9631c006-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.276600 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.276463 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/491953d0-3e2f-4b08-aacf-316d9631c006-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.277126 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.276921 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/491953d0-3e2f-4b08-aacf-316d9631c006-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.279275 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.278906 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/491953d0-3e2f-4b08-aacf-316d9631c006-config-out\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.279275 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.279034 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/491953d0-3e2f-4b08-aacf-316d9631c006-tls-assets\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.279275 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.279052 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.279275 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.279155 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-web-config\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.279275 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.279223 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.279636 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.279284 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-config-volume\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.279636 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.279323 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.279636 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.279564 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.280136 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.280113 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491953d0-3e2f-4b08-aacf-316d9631c006-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.280804 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.280783 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/491953d0-3e2f-4b08-aacf-316d9631c006-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.291424 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.291391 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plz67\" (UniqueName: \"kubernetes.io/projected/491953d0-3e2f-4b08-aacf-316d9631c006-kube-api-access-plz67\") pod \"alertmanager-main-0\" (UID: \"491953d0-3e2f-4b08-aacf-316d9631c006\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.367531 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.367489 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 12:12:39.479855 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.479824 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28a3e2f1-66bb-41f2-afe1-79c2b8e67595" path="/var/lib/kubelet/pods/28a3e2f1-66bb-41f2-afe1-79c2b8e67595/volumes" Jul 16 12:12:39.530007 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.528760 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 12:12:39.678564 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.678529 2578 generic.go:356] "Generic (PLEG): container finished" podID="491953d0-3e2f-4b08-aacf-316d9631c006" containerID="008863617f00d1686406aa85f0204bfcb883131139aff6bf89bc67ae28c29bfc" exitCode=0 Jul 16 12:12:39.678977 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.678607 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"491953d0-3e2f-4b08-aacf-316d9631c006","Type":"ContainerDied","Data":"008863617f00d1686406aa85f0204bfcb883131139aff6bf89bc67ae28c29bfc"} Jul 16 12:12:39.678977 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.678686 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"491953d0-3e2f-4b08-aacf-316d9631c006","Type":"ContainerStarted","Data":"173dff74759cf22a976f43cc81155c8317731417ae96684499a564f2d0047dc6"} Jul 16 12:12:39.787003 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.786925 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:39.887339 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:39.887306 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:40.689859 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:40.689815 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"491953d0-3e2f-4b08-aacf-316d9631c006","Type":"ContainerStarted","Data":"ab5c35627958390c81eeb9c1dcb819934617aeb12dff160079a1722343dcf75e"} Jul 16 12:12:40.690299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:40.689879 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"491953d0-3e2f-4b08-aacf-316d9631c006","Type":"ContainerStarted","Data":"f5b5e25f1a19ec085067f27efdcfe9e85f18bd8e5ac01cf7a436fa2bf9e5f037"} Jul 16 12:12:40.690299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:40.689904 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"491953d0-3e2f-4b08-aacf-316d9631c006","Type":"ContainerStarted","Data":"8dcf0238e0ad44d3792f04ce053f3baede39764f62adef0e8092f321d30af3ef"} Jul 16 12:12:40.690299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:40.689929 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"491953d0-3e2f-4b08-aacf-316d9631c006","Type":"ContainerStarted","Data":"46b76780b1f581a77c34238e06f50505ca3ea90529ec096945db10ee43751b2a"} Jul 16 12:12:40.690299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:40.689960 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"491953d0-3e2f-4b08-aacf-316d9631c006","Type":"ContainerStarted","Data":"a32a0e97d7c8a2dbc00ae4bb4bf0672e7edb165d9296cd4b81ca3dfaff3a8d80"} Jul 16 12:12:40.690299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:40.689983 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"491953d0-3e2f-4b08-aacf-316d9631c006","Type":"ContainerStarted","Data":"25d7ba1b042bf41e4c87517f9f80936b5f27b463be9d73c082ea253caff7d7a0"} Jul 16 12:12:40.727252 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:40.727197 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.711627869 podStartE2EDuration="1.727182325s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.554456ms" isStatefulPod=false podCreationTimestamp="2026-07-16 12:12:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:12:40.726294778 +0000 UTC m=+310.489664277" watchObservedRunningTime="2026-07-16 12:12:40.727182325 +0000 UTC m=+310.490551819" Jul 16 12:12:45.943476 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:45.943406 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:46.043477 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:46.043437 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:47.700387 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:47.700352 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:47.700778 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:47.700448 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:47.705299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:47.705277 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:47.730540 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:47.730497 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:12:47.802561 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:47.802522 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7447fd5c77-qt5gr"] Jul 16 12:12:48.241474 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:48.241398 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:48.341581 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:48.341544 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:58.874045 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:58.874002 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:58.979859 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:58.979817 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:12:59.081294 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:12:59.081257 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:13:12.826241 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:12.826194 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7447fd5c77-qt5gr" podUID="79799e3b-f6d0-4a23-8e07-718e984d761b" containerName="console" containerID="cri-o://e894828700f7a12b56688009f63b4045aa96958d4f8fd48368e449cf581ee689" gracePeriod=15 Jul 16 12:13:13.065869 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.065844 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:13:13.163370 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.163277 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-oauth-serving-cert\") pod \"79799e3b-f6d0-4a23-8e07-718e984d761b\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " Jul 16 12:13:13.163370 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.163335 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vs58\" (UniqueName: \"kubernetes.io/projected/79799e3b-f6d0-4a23-8e07-718e984d761b-kube-api-access-4vs58\") pod \"79799e3b-f6d0-4a23-8e07-718e984d761b\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " Jul 16 12:13:13.163370 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.163372 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79799e3b-f6d0-4a23-8e07-718e984d761b-console-oauth-config\") pod \"79799e3b-f6d0-4a23-8e07-718e984d761b\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " Jul 16 12:13:13.163666 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.163389 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-service-ca\") pod \"79799e3b-f6d0-4a23-8e07-718e984d761b\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " Jul 16 12:13:13.163666 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.163464 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79799e3b-f6d0-4a23-8e07-718e984d761b-console-serving-cert\") pod \"79799e3b-f6d0-4a23-8e07-718e984d761b\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " Jul 16 12:13:13.163666 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.163521 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-trusted-ca-bundle\") pod \"79799e3b-f6d0-4a23-8e07-718e984d761b\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " Jul 16 12:13:13.163666 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.163545 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-console-config\") pod \"79799e3b-f6d0-4a23-8e07-718e984d761b\" (UID: \"79799e3b-f6d0-4a23-8e07-718e984d761b\") " Jul 16 12:13:13.163867 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.163686 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "79799e3b-f6d0-4a23-8e07-718e984d761b" (UID: "79799e3b-f6d0-4a23-8e07-718e984d761b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:13:13.163867 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.163850 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-oauth-serving-cert\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:13:13.164000 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.163965 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "79799e3b-f6d0-4a23-8e07-718e984d761b" (UID: "79799e3b-f6d0-4a23-8e07-718e984d761b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:13:13.164134 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.164032 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-service-ca" (OuterVolumeSpecName: "service-ca") pod "79799e3b-f6d0-4a23-8e07-718e984d761b" (UID: "79799e3b-f6d0-4a23-8e07-718e984d761b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:13:13.164134 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.164028 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-console-config" (OuterVolumeSpecName: "console-config") pod "79799e3b-f6d0-4a23-8e07-718e984d761b" (UID: "79799e3b-f6d0-4a23-8e07-718e984d761b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:13:13.165566 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.165542 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79799e3b-f6d0-4a23-8e07-718e984d761b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "79799e3b-f6d0-4a23-8e07-718e984d761b" (UID: "79799e3b-f6d0-4a23-8e07-718e984d761b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:13:13.165644 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.165626 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79799e3b-f6d0-4a23-8e07-718e984d761b-kube-api-access-4vs58" (OuterVolumeSpecName: "kube-api-access-4vs58") pod "79799e3b-f6d0-4a23-8e07-718e984d761b" (UID: "79799e3b-f6d0-4a23-8e07-718e984d761b"). InnerVolumeSpecName "kube-api-access-4vs58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:13:13.165644 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.165629 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79799e3b-f6d0-4a23-8e07-718e984d761b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "79799e3b-f6d0-4a23-8e07-718e984d761b" (UID: "79799e3b-f6d0-4a23-8e07-718e984d761b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:13:13.264708 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.264677 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vs58\" (UniqueName: \"kubernetes.io/projected/79799e3b-f6d0-4a23-8e07-718e984d761b-kube-api-access-4vs58\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:13:13.264708 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.264703 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79799e3b-f6d0-4a23-8e07-718e984d761b-console-oauth-config\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:13:13.264708 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.264712 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-service-ca\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:13:13.264912 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.264721 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79799e3b-f6d0-4a23-8e07-718e984d761b-console-serving-cert\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:13:13.264912 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.264730 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-trusted-ca-bundle\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:13:13.264912 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.264738 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79799e3b-f6d0-4a23-8e07-718e984d761b-console-config\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:13:13.841080 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.841047 2578 generic.go:356] "Generic (PLEG): container finished" podID="79799e3b-f6d0-4a23-8e07-718e984d761b" containerID="e894828700f7a12b56688009f63b4045aa96958d4f8fd48368e449cf581ee689" exitCode=0 Jul 16 12:13:13.841524 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.841088 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7447fd5c77-qt5gr" Jul 16 12:13:13.841524 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.841087 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7447fd5c77-qt5gr" event={"ID":"79799e3b-f6d0-4a23-8e07-718e984d761b","Type":"ContainerDied","Data":"e894828700f7a12b56688009f63b4045aa96958d4f8fd48368e449cf581ee689"} Jul 16 12:13:13.841524 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.841239 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7447fd5c77-qt5gr" event={"ID":"79799e3b-f6d0-4a23-8e07-718e984d761b","Type":"ContainerDied","Data":"0477edfddab3281635505c3b9f9a9d5379fdb24579568dfed42ef6b0fa129847"} Jul 16 12:13:13.841524 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.841275 2578 scope.go:117] "RemoveContainer" containerID="e894828700f7a12b56688009f63b4045aa96958d4f8fd48368e449cf581ee689" Jul 16 12:13:13.849337 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.849322 2578 scope.go:117] "RemoveContainer" containerID="e894828700f7a12b56688009f63b4045aa96958d4f8fd48368e449cf581ee689" Jul 16 12:13:13.849628 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:13:13.849606 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e894828700f7a12b56688009f63b4045aa96958d4f8fd48368e449cf581ee689\": container with ID starting with e894828700f7a12b56688009f63b4045aa96958d4f8fd48368e449cf581ee689 not found: ID does not exist" containerID="e894828700f7a12b56688009f63b4045aa96958d4f8fd48368e449cf581ee689" Jul 16 12:13:13.849722 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.849633 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e894828700f7a12b56688009f63b4045aa96958d4f8fd48368e449cf581ee689"} err="failed to get container status \"e894828700f7a12b56688009f63b4045aa96958d4f8fd48368e449cf581ee689\": rpc error: code = NotFound desc = could not find container \"e894828700f7a12b56688009f63b4045aa96958d4f8fd48368e449cf581ee689\": container with ID starting with e894828700f7a12b56688009f63b4045aa96958d4f8fd48368e449cf581ee689 not found: ID does not exist" Jul 16 12:13:13.866821 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.866797 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7447fd5c77-qt5gr"] Jul 16 12:13:13.869752 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:13.869731 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7447fd5c77-qt5gr"] Jul 16 12:13:15.478602 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:15.478568 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79799e3b-f6d0-4a23-8e07-718e984d761b" path="/var/lib/kubelet/pods/79799e3b-f6d0-4a23-8e07-718e984d761b/volumes" Jul 16 12:13:39.390688 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:39.390653 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:13:39.470837 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:39.470798 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:13:49.335830 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:49.335790 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:13:49.418536 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:49.418494 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:13:50.142406 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.142377 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz"] Jul 16 12:13:50.142721 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.142709 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79799e3b-f6d0-4a23-8e07-718e984d761b" containerName="console" Jul 16 12:13:50.142767 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.142722 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="79799e3b-f6d0-4a23-8e07-718e984d761b" containerName="console" Jul 16 12:13:50.142804 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.142776 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="79799e3b-f6d0-4a23-8e07-718e984d761b" containerName="console" Jul 16 12:13:50.147280 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.147265 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" Jul 16 12:13:50.150135 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.150111 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nkzbt\"" Jul 16 12:13:50.150237 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.150152 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 12:13:50.151658 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.151642 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 12:13:50.160788 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.160767 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz"] Jul 16 12:13:50.189362 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.189335 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45wqv\" (UniqueName: \"kubernetes.io/projected/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-kube-api-access-45wqv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz\" (UID: \"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" Jul 16 12:13:50.189475 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.189378 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz\" (UID: \"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" Jul 16 12:13:50.189518 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.189480 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz\" (UID: \"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" Jul 16 12:13:50.290350 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.290314 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz\" (UID: \"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" Jul 16 12:13:50.290540 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.290365 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz\" (UID: \"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" Jul 16 12:13:50.290540 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.290452 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45wqv\" (UniqueName: \"kubernetes.io/projected/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-kube-api-access-45wqv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz\" (UID: \"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" Jul 16 12:13:50.290703 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.290684 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz\" (UID: \"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" Jul 16 12:13:50.290768 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.290748 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz\" (UID: \"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" Jul 16 12:13:50.299649 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.299629 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45wqv\" (UniqueName: \"kubernetes.io/projected/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-kube-api-access-45wqv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz\" (UID: \"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" Jul 16 12:13:50.456523 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.456431 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" Jul 16 12:13:50.590313 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.590291 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz"] Jul 16 12:13:50.592605 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:13:50.592578 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e5e2d2_15e0_4e5e_9d89_28ec83b35ac8.slice/crio-f7e0fe761422cadd1bbea24215efc115e8c8f96db12fa1de656c321a6dcd8cad WatchSource:0}: Error finding container f7e0fe761422cadd1bbea24215efc115e8c8f96db12fa1de656c321a6dcd8cad: Status 404 returned error can't find the container with id f7e0fe761422cadd1bbea24215efc115e8c8f96db12fa1de656c321a6dcd8cad Jul 16 12:13:50.594596 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.594577 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 12:13:50.994005 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:50.993971 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" event={"ID":"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8","Type":"ContainerStarted","Data":"f7e0fe761422cadd1bbea24215efc115e8c8f96db12fa1de656c321a6dcd8cad"} Jul 16 12:13:57.027217 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:57.027128 2578 generic.go:356] "Generic (PLEG): container finished" podID="51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8" containerID="9292f3677c2dc8916792ba6a6e2ce697f8873f3242b4a3987aa53e6784bf8e96" exitCode=0 Jul 16 12:13:57.027640 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:13:57.027209 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" event={"ID":"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8","Type":"ContainerDied","Data":"9292f3677c2dc8916792ba6a6e2ce697f8873f3242b4a3987aa53e6784bf8e96"} Jul 16 12:14:00.042658 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:00.042626 2578 generic.go:356] "Generic (PLEG): container finished" podID="51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8" containerID="c51d3c61aaa6f4c87bf62dcac54f95d39653913625bc5bfc3c1191105396302b" exitCode=0 Jul 16 12:14:00.043041 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:00.042725 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" event={"ID":"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8","Type":"ContainerDied","Data":"c51d3c61aaa6f4c87bf62dcac54f95d39653913625bc5bfc3c1191105396302b"} Jul 16 12:14:09.087611 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:09.087576 2578 generic.go:356] "Generic (PLEG): container finished" podID="51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8" containerID="268c81e6565c307963caff00fbe0e4b9046aff0198f0936a5f848d738a556364" exitCode=0 Jul 16 12:14:09.087996 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:09.087661 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" event={"ID":"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8","Type":"ContainerDied","Data":"268c81e6565c307963caff00fbe0e4b9046aff0198f0936a5f848d738a556364"} Jul 16 12:14:10.220912 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:10.220891 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" Jul 16 12:14:10.380588 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:10.380484 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-util\") pod \"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8\" (UID: \"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8\") " Jul 16 12:14:10.380588 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:10.380539 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-bundle\") pod \"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8\" (UID: \"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8\") " Jul 16 12:14:10.380588 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:10.380569 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45wqv\" (UniqueName: \"kubernetes.io/projected/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-kube-api-access-45wqv\") pod \"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8\" (UID: \"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8\") " Jul 16 12:14:10.381230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:10.381196 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-bundle" (OuterVolumeSpecName: "bundle") pod "51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8" (UID: "51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:14:10.382740 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:10.382709 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-kube-api-access-45wqv" (OuterVolumeSpecName: "kube-api-access-45wqv") pod "51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8" (UID: "51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8"). InnerVolumeSpecName "kube-api-access-45wqv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:14:10.384495 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:10.384475 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-util" (OuterVolumeSpecName: "util") pod "51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8" (UID: "51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:14:10.481305 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:10.481271 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-util\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:14:10.481305 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:10.481299 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-bundle\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:14:10.481305 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:10.481308 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45wqv\" (UniqueName: \"kubernetes.io/projected/51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8-kube-api-access-45wqv\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:14:11.096828 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:11.096798 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" Jul 16 12:14:11.096828 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:11.096805 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540skrsz" event={"ID":"51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8","Type":"ContainerDied","Data":"f7e0fe761422cadd1bbea24215efc115e8c8f96db12fa1de656c321a6dcd8cad"} Jul 16 12:14:11.097027 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:11.096847 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7e0fe761422cadd1bbea24215efc115e8c8f96db12fa1de656c321a6dcd8cad" Jul 16 12:14:16.274493 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:16.274456 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:17.946367 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:17.946328 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:19.868610 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:19.868572 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nj4m"] Jul 16 12:14:19.869003 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:19.868885 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8" containerName="pull" Jul 16 12:14:19.869003 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:19.868896 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8" containerName="pull" Jul 16 12:14:19.869003 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:19.868908 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8" containerName="util" Jul 16 12:14:19.869003 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:19.868913 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8" containerName="util" Jul 16 12:14:19.869003 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:19.868927 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8" containerName="extract" Jul 16 12:14:19.869003 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:19.868933 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8" containerName="extract" Jul 16 12:14:19.869003 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:19.868985 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="51e5e2d2-15e0-4e5e-9d89-28ec83b35ac8" containerName="extract" Jul 16 12:14:19.870956 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:19.870939 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nj4m" Jul 16 12:14:19.878082 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:19.878057 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-8snnn\"" Jul 16 12:14:19.878194 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:19.878165 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 16 12:14:19.878876 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:19.878856 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 16 12:14:19.933073 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:19.933044 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nj4m"] Jul 16 12:14:19.971151 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:19.971126 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17e9cc2c-42c6-4e69-bdee-1da8534a47d7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6nj4m\" (UID: \"17e9cc2c-42c6-4e69-bdee-1da8534a47d7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nj4m" Jul 16 12:14:19.971277 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:19.971162 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmtkb\" (UniqueName: \"kubernetes.io/projected/17e9cc2c-42c6-4e69-bdee-1da8534a47d7-kube-api-access-lmtkb\") pod \"cert-manager-operator-controller-manager-55c444bf86-6nj4m\" (UID: \"17e9cc2c-42c6-4e69-bdee-1da8534a47d7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nj4m" Jul 16 12:14:20.072129 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:20.072095 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmtkb\" (UniqueName: \"kubernetes.io/projected/17e9cc2c-42c6-4e69-bdee-1da8534a47d7-kube-api-access-lmtkb\") pod \"cert-manager-operator-controller-manager-55c444bf86-6nj4m\" (UID: \"17e9cc2c-42c6-4e69-bdee-1da8534a47d7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nj4m" Jul 16 12:14:20.072288 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:20.072195 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17e9cc2c-42c6-4e69-bdee-1da8534a47d7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6nj4m\" (UID: \"17e9cc2c-42c6-4e69-bdee-1da8534a47d7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nj4m" Jul 16 12:14:20.072568 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:20.072552 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17e9cc2c-42c6-4e69-bdee-1da8534a47d7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6nj4m\" (UID: \"17e9cc2c-42c6-4e69-bdee-1da8534a47d7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nj4m" Jul 16 12:14:20.090236 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:20.090207 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmtkb\" (UniqueName: \"kubernetes.io/projected/17e9cc2c-42c6-4e69-bdee-1da8534a47d7-kube-api-access-lmtkb\") pod \"cert-manager-operator-controller-manager-55c444bf86-6nj4m\" (UID: \"17e9cc2c-42c6-4e69-bdee-1da8534a47d7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nj4m" Jul 16 12:14:20.180014 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:20.179932 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nj4m" Jul 16 12:14:20.330955 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:20.330934 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nj4m"] Jul 16 12:14:20.333101 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:14:20.333064 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17e9cc2c_42c6_4e69_bdee_1da8534a47d7.slice/crio-8fa6584fd0cda56ec042d69c3eec87dbad07098ea961c9048250b72a6b395607 WatchSource:0}: Error finding container 8fa6584fd0cda56ec042d69c3eec87dbad07098ea961c9048250b72a6b395607: Status 404 returned error can't find the container with id 8fa6584fd0cda56ec042d69c3eec87dbad07098ea961c9048250b72a6b395607 Jul 16 12:14:21.141147 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:21.141108 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nj4m" event={"ID":"17e9cc2c-42c6-4e69-bdee-1da8534a47d7","Type":"ContainerStarted","Data":"8fa6584fd0cda56ec042d69c3eec87dbad07098ea961c9048250b72a6b395607"} Jul 16 12:14:21.278337 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:21.278297 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:21.413896 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:21.413813 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:23.085023 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:23.084988 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:23.152576 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:23.152543 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nj4m" event={"ID":"17e9cc2c-42c6-4e69-bdee-1da8534a47d7","Type":"ContainerStarted","Data":"c29e9da4527b79a30e1325c4e65cf7d9d184fee924b59c2b88a44bf968661460"} Jul 16 12:14:23.196400 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:23.196364 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:23.217447 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:23.214383 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nj4m" podStartSLOduration=2.438843102 podStartE2EDuration="4.214362223s" totalImagesPullingTime="1.775519121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:14:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:14:23.213964695 +0000 UTC m=+412.977334158" watchObservedRunningTime="2026-07-16 12:14:23.214362223 +0000 UTC m=+412.977731688" Jul 16 12:14:23.798562 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:23.798524 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:23.929059 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:23.929020 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:24.102982 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.102901 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx"] Jul 16 12:14:24.106383 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.106365 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" Jul 16 12:14:24.107096 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.107076 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e008eb16-7876-4213-913e-605b5e5e23fe-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx\" (UID: \"e008eb16-7876-4213-913e-605b5e5e23fe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" Jul 16 12:14:24.107231 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.107107 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm5gs\" (UniqueName: \"kubernetes.io/projected/e008eb16-7876-4213-913e-605b5e5e23fe-kube-api-access-mm5gs\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx\" (UID: \"e008eb16-7876-4213-913e-605b5e5e23fe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" Jul 16 12:14:24.107231 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.107206 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e008eb16-7876-4213-913e-605b5e5e23fe-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx\" (UID: \"e008eb16-7876-4213-913e-605b5e5e23fe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" Jul 16 12:14:24.111848 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.111829 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nkzbt\"" Jul 16 12:14:24.111947 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.111827 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 12:14:24.112603 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.112585 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 12:14:24.125774 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.125750 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx"] Jul 16 12:14:24.208132 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.208096 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e008eb16-7876-4213-913e-605b5e5e23fe-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx\" (UID: \"e008eb16-7876-4213-913e-605b5e5e23fe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" Jul 16 12:14:24.208321 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.208138 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm5gs\" (UniqueName: \"kubernetes.io/projected/e008eb16-7876-4213-913e-605b5e5e23fe-kube-api-access-mm5gs\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx\" (UID: \"e008eb16-7876-4213-913e-605b5e5e23fe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" Jul 16 12:14:24.208321 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.208172 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e008eb16-7876-4213-913e-605b5e5e23fe-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx\" (UID: \"e008eb16-7876-4213-913e-605b5e5e23fe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" Jul 16 12:14:24.208586 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.208566 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e008eb16-7876-4213-913e-605b5e5e23fe-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx\" (UID: \"e008eb16-7876-4213-913e-605b5e5e23fe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" Jul 16 12:14:24.208652 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.208582 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e008eb16-7876-4213-913e-605b5e5e23fe-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx\" (UID: \"e008eb16-7876-4213-913e-605b5e5e23fe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" Jul 16 12:14:24.250062 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.250027 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm5gs\" (UniqueName: \"kubernetes.io/projected/e008eb16-7876-4213-913e-605b5e5e23fe-kube-api-access-mm5gs\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx\" (UID: \"e008eb16-7876-4213-913e-605b5e5e23fe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" Jul 16 12:14:24.415741 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.415659 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" Jul 16 12:14:24.563372 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:24.563338 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx"] Jul 16 12:14:24.578298 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:14:24.578272 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode008eb16_7876_4213_913e_605b5e5e23fe.slice/crio-16cd919fc4c09bad8c2a5c84b0a8d1a2c93f40a453495132438cad1b810ef72a WatchSource:0}: Error finding container 16cd919fc4c09bad8c2a5c84b0a8d1a2c93f40a453495132438cad1b810ef72a: Status 404 returned error can't find the container with id 16cd919fc4c09bad8c2a5c84b0a8d1a2c93f40a453495132438cad1b810ef72a Jul 16 12:14:25.164905 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:25.164868 2578 generic.go:356] "Generic (PLEG): container finished" podID="e008eb16-7876-4213-913e-605b5e5e23fe" containerID="f25a9892f8bb2cff9488b64a8d9fdc659a4a38f5724862cc2b425f2257d0486a" exitCode=0 Jul 16 12:14:25.165379 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:25.164941 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" event={"ID":"e008eb16-7876-4213-913e-605b5e5e23fe","Type":"ContainerDied","Data":"f25a9892f8bb2cff9488b64a8d9fdc659a4a38f5724862cc2b425f2257d0486a"} Jul 16 12:14:25.165379 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:25.164985 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" event={"ID":"e008eb16-7876-4213-913e-605b5e5e23fe","Type":"ContainerStarted","Data":"16cd919fc4c09bad8c2a5c84b0a8d1a2c93f40a453495132438cad1b810ef72a"} Jul 16 12:14:25.820007 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:25.819965 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:25.968788 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:25.968745 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:28.180536 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:28.180501 2578 generic.go:356] "Generic (PLEG): container finished" podID="e008eb16-7876-4213-913e-605b5e5e23fe" containerID="926ee924bea6a3bfa46d6dd78c9de27d62b93f9fd53080a6f2ce852d08a23aaa" exitCode=0 Jul 16 12:14:28.180969 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:28.180583 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" event={"ID":"e008eb16-7876-4213-913e-605b5e5e23fe","Type":"ContainerDied","Data":"926ee924bea6a3bfa46d6dd78c9de27d62b93f9fd53080a6f2ce852d08a23aaa"} Jul 16 12:14:29.187263 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:29.187225 2578 generic.go:356] "Generic (PLEG): container finished" podID="e008eb16-7876-4213-913e-605b5e5e23fe" containerID="8df4e7e5b617cdb1abdf4be7a9846329cfe9e78c3aa7dbf51b27283d30fcc9b1" exitCode=0 Jul 16 12:14:29.187650 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:29.187300 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" event={"ID":"e008eb16-7876-4213-913e-605b5e5e23fe","Type":"ContainerDied","Data":"8df4e7e5b617cdb1abdf4be7a9846329cfe9e78c3aa7dbf51b27283d30fcc9b1"} Jul 16 12:14:30.357677 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:30.357646 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" Jul 16 12:14:30.455870 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:30.455794 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mm5gs\" (UniqueName: \"kubernetes.io/projected/e008eb16-7876-4213-913e-605b5e5e23fe-kube-api-access-mm5gs\") pod \"e008eb16-7876-4213-913e-605b5e5e23fe\" (UID: \"e008eb16-7876-4213-913e-605b5e5e23fe\") " Jul 16 12:14:30.455870 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:30.455846 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e008eb16-7876-4213-913e-605b5e5e23fe-util\") pod \"e008eb16-7876-4213-913e-605b5e5e23fe\" (UID: \"e008eb16-7876-4213-913e-605b5e5e23fe\") " Jul 16 12:14:30.456039 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:30.455961 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e008eb16-7876-4213-913e-605b5e5e23fe-bundle\") pod \"e008eb16-7876-4213-913e-605b5e5e23fe\" (UID: \"e008eb16-7876-4213-913e-605b5e5e23fe\") " Jul 16 12:14:30.456333 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:30.456296 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e008eb16-7876-4213-913e-605b5e5e23fe-bundle" (OuterVolumeSpecName: "bundle") pod "e008eb16-7876-4213-913e-605b5e5e23fe" (UID: "e008eb16-7876-4213-913e-605b5e5e23fe"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:14:30.457926 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:30.457899 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e008eb16-7876-4213-913e-605b5e5e23fe-kube-api-access-mm5gs" (OuterVolumeSpecName: "kube-api-access-mm5gs") pod "e008eb16-7876-4213-913e-605b5e5e23fe" (UID: "e008eb16-7876-4213-913e-605b5e5e23fe"). InnerVolumeSpecName "kube-api-access-mm5gs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:14:30.460721 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:30.460680 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e008eb16-7876-4213-913e-605b5e5e23fe-util" (OuterVolumeSpecName: "util") pod "e008eb16-7876-4213-913e-605b5e5e23fe" (UID: "e008eb16-7876-4213-913e-605b5e5e23fe"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:14:30.556956 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:30.556925 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e008eb16-7876-4213-913e-605b5e5e23fe-util\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:14:30.556956 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:30.556949 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e008eb16-7876-4213-913e-605b5e5e23fe-bundle\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:14:30.557114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:30.556963 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm5gs\" (UniqueName: \"kubernetes.io/projected/e008eb16-7876-4213-913e-605b5e5e23fe-kube-api-access-mm5gs\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:14:31.199467 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:31.199425 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" event={"ID":"e008eb16-7876-4213-913e-605b5e5e23fe","Type":"ContainerDied","Data":"16cd919fc4c09bad8c2a5c84b0a8d1a2c93f40a453495132438cad1b810ef72a"} Jul 16 12:14:31.199467 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:31.199464 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffcdtx" Jul 16 12:14:31.199693 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:31.199481 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16cd919fc4c09bad8c2a5c84b0a8d1a2c93f40a453495132438cad1b810ef72a" Jul 16 12:14:32.707803 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:32.707740 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:32.877378 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:32.877329 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:33.802916 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.802882 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j5898"] Jul 16 12:14:33.803299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.803215 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e008eb16-7876-4213-913e-605b5e5e23fe" containerName="util" Jul 16 12:14:33.803299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.803228 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e008eb16-7876-4213-913e-605b5e5e23fe" containerName="util" Jul 16 12:14:33.803299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.803243 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e008eb16-7876-4213-913e-605b5e5e23fe" containerName="pull" Jul 16 12:14:33.803299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.803249 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e008eb16-7876-4213-913e-605b5e5e23fe" containerName="pull" Jul 16 12:14:33.803299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.803260 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e008eb16-7876-4213-913e-605b5e5e23fe" containerName="extract" Jul 16 12:14:33.803299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.803266 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e008eb16-7876-4213-913e-605b5e5e23fe" containerName="extract" Jul 16 12:14:33.803507 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.803341 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e008eb16-7876-4213-913e-605b5e5e23fe" containerName="extract" Jul 16 12:14:33.805613 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.805591 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-j5898" Jul 16 12:14:33.808604 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.808580 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 16 12:14:33.808718 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.808700 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 16 12:14:33.809573 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.809552 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-qpckr\"" Jul 16 12:14:33.818599 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.818572 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j5898"] Jul 16 12:14:33.884831 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.884796 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89877275-c6e6-45c3-ac49-4781faf1cf8d-bound-sa-token\") pod \"cert-manager-55b4d954d5-j5898\" (UID: \"89877275-c6e6-45c3-ac49-4781faf1cf8d\") " pod="cert-manager/cert-manager-55b4d954d5-j5898" Jul 16 12:14:33.884831 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.884839 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8pbx\" (UniqueName: \"kubernetes.io/projected/89877275-c6e6-45c3-ac49-4781faf1cf8d-kube-api-access-d8pbx\") pod \"cert-manager-55b4d954d5-j5898\" (UID: \"89877275-c6e6-45c3-ac49-4781faf1cf8d\") " pod="cert-manager/cert-manager-55b4d954d5-j5898" Jul 16 12:14:33.986171 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.986133 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89877275-c6e6-45c3-ac49-4781faf1cf8d-bound-sa-token\") pod \"cert-manager-55b4d954d5-j5898\" (UID: \"89877275-c6e6-45c3-ac49-4781faf1cf8d\") " pod="cert-manager/cert-manager-55b4d954d5-j5898" Jul 16 12:14:33.986346 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.986196 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8pbx\" (UniqueName: \"kubernetes.io/projected/89877275-c6e6-45c3-ac49-4781faf1cf8d-kube-api-access-d8pbx\") pod \"cert-manager-55b4d954d5-j5898\" (UID: \"89877275-c6e6-45c3-ac49-4781faf1cf8d\") " pod="cert-manager/cert-manager-55b4d954d5-j5898" Jul 16 12:14:33.995598 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.995567 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89877275-c6e6-45c3-ac49-4781faf1cf8d-bound-sa-token\") pod \"cert-manager-55b4d954d5-j5898\" (UID: \"89877275-c6e6-45c3-ac49-4781faf1cf8d\") " pod="cert-manager/cert-manager-55b4d954d5-j5898" Jul 16 12:14:33.995733 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:33.995698 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8pbx\" (UniqueName: \"kubernetes.io/projected/89877275-c6e6-45c3-ac49-4781faf1cf8d-kube-api-access-d8pbx\") pod \"cert-manager-55b4d954d5-j5898\" (UID: \"89877275-c6e6-45c3-ac49-4781faf1cf8d\") " pod="cert-manager/cert-manager-55b4d954d5-j5898" Jul 16 12:14:34.130117 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:34.130036 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-j5898" Jul 16 12:14:34.283131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:34.283099 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j5898"] Jul 16 12:14:34.285738 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:14:34.285710 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89877275_c6e6_45c3_ac49_4781faf1cf8d.slice/crio-2f4fab655937e10c6d2c2426a49811ab97589345812d1886469bc37eee45b781 WatchSource:0}: Error finding container 2f4fab655937e10c6d2c2426a49811ab97589345812d1886469bc37eee45b781: Status 404 returned error can't find the container with id 2f4fab655937e10c6d2c2426a49811ab97589345812d1886469bc37eee45b781 Jul 16 12:14:34.583124 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:34.583068 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:34.686308 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:34.686262 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:35.221389 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:35.221348 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-j5898" event={"ID":"89877275-c6e6-45c3-ac49-4781faf1cf8d","Type":"ContainerStarted","Data":"2f4fab655937e10c6d2c2426a49811ab97589345812d1886469bc37eee45b781"} Jul 16 12:14:35.884941 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:35.884907 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lqffn"] Jul 16 12:14:35.887366 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:35.887341 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lqffn" Jul 16 12:14:35.890114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:35.890093 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 16 12:14:35.890269 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:35.890099 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 16 12:14:35.890914 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:35.890874 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-5kkwj\"" Jul 16 12:14:35.901245 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:35.901221 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c36a8149-0d59-4753-bf82-af6b60323e61-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lqffn\" (UID: \"c36a8149-0d59-4753-bf82-af6b60323e61\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lqffn" Jul 16 12:14:35.901359 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:35.901336 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n6r2\" (UniqueName: \"kubernetes.io/projected/c36a8149-0d59-4753-bf82-af6b60323e61-kube-api-access-4n6r2\") pod \"openshift-lws-operator-fd8ccff4c-lqffn\" (UID: \"c36a8149-0d59-4753-bf82-af6b60323e61\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lqffn" Jul 16 12:14:35.902599 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:35.902577 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lqffn"] Jul 16 12:14:36.002341 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:36.002303 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n6r2\" (UniqueName: \"kubernetes.io/projected/c36a8149-0d59-4753-bf82-af6b60323e61-kube-api-access-4n6r2\") pod \"openshift-lws-operator-fd8ccff4c-lqffn\" (UID: \"c36a8149-0d59-4753-bf82-af6b60323e61\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lqffn" Jul 16 12:14:36.002529 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:36.002381 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c36a8149-0d59-4753-bf82-af6b60323e61-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lqffn\" (UID: \"c36a8149-0d59-4753-bf82-af6b60323e61\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lqffn" Jul 16 12:14:36.002816 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:36.002794 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c36a8149-0d59-4753-bf82-af6b60323e61-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lqffn\" (UID: \"c36a8149-0d59-4753-bf82-af6b60323e61\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lqffn" Jul 16 12:14:36.013126 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:36.013099 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n6r2\" (UniqueName: \"kubernetes.io/projected/c36a8149-0d59-4753-bf82-af6b60323e61-kube-api-access-4n6r2\") pod \"openshift-lws-operator-fd8ccff4c-lqffn\" (UID: \"c36a8149-0d59-4753-bf82-af6b60323e61\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lqffn" Jul 16 12:14:36.199163 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:36.199081 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lqffn" Jul 16 12:14:36.702258 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:36.702233 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lqffn"] Jul 16 12:14:36.704769 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:14:36.704745 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc36a8149_0d59_4753_bf82_af6b60323e61.slice/crio-2c28516625e715067e302f29af7d3ec8df31734653787bea17f99148f2243305 WatchSource:0}: Error finding container 2c28516625e715067e302f29af7d3ec8df31734653787bea17f99148f2243305: Status 404 returned error can't find the container with id 2c28516625e715067e302f29af7d3ec8df31734653787bea17f99148f2243305 Jul 16 12:14:37.232769 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:37.232732 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-j5898" event={"ID":"89877275-c6e6-45c3-ac49-4781faf1cf8d","Type":"ContainerStarted","Data":"689eccca0bac62607ca256f45928266f153dbcbfa5bffdbed0964fc0932557a7"} Jul 16 12:14:37.234079 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:37.234053 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lqffn" event={"ID":"c36a8149-0d59-4753-bf82-af6b60323e61","Type":"ContainerStarted","Data":"2c28516625e715067e302f29af7d3ec8df31734653787bea17f99148f2243305"} Jul 16 12:14:37.253858 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:37.253810 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-j5898" podStartSLOduration=1.934801121 podStartE2EDuration="4.253793561s" totalImagesPullingTime="2.31899244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:14:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:14:37.252253342 +0000 UTC m=+427.015622806" watchObservedRunningTime="2026-07-16 12:14:37.253793561 +0000 UTC m=+427.017163023" Jul 16 12:14:39.247527 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:39.247494 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lqffn" event={"ID":"c36a8149-0d59-4753-bf82-af6b60323e61","Type":"ContainerStarted","Data":"76449d34e1e2980e4594fc31066d425a2bbbdaecaa561d11cd4cc639fa1b92fb"} Jul 16 12:14:39.270653 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:39.270595 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lqffn" podStartSLOduration=2.612112739 podStartE2EDuration="4.270578355s" totalImagesPullingTime="1.658465616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:14:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:14:39.269990813 +0000 UTC m=+429.033360287" watchObservedRunningTime="2026-07-16 12:14:39.270578355 +0000 UTC m=+429.033947817" Jul 16 12:14:40.246272 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:40.246235 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:40.346321 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:40.346283 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:41.703162 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:41.703131 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq"] Jul 16 12:14:41.726104 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:41.726079 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" Jul 16 12:14:41.728906 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:41.728880 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nkzbt\"" Jul 16 12:14:41.729051 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:41.729016 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 12:14:41.729747 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:41.729725 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 12:14:41.729869 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:41.729818 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq"] Jul 16 12:14:41.754432 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:41.754376 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/683bce1b-f61f-428b-ad81-fae3e9f82945-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq\" (UID: \"683bce1b-f61f-428b-ad81-fae3e9f82945\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" Jul 16 12:14:41.754583 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:41.754452 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/683bce1b-f61f-428b-ad81-fae3e9f82945-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq\" (UID: \"683bce1b-f61f-428b-ad81-fae3e9f82945\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" Jul 16 12:14:41.754583 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:41.754498 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssc4x\" (UniqueName: \"kubernetes.io/projected/683bce1b-f61f-428b-ad81-fae3e9f82945-kube-api-access-ssc4x\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq\" (UID: \"683bce1b-f61f-428b-ad81-fae3e9f82945\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" Jul 16 12:14:41.855397 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:41.855353 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/683bce1b-f61f-428b-ad81-fae3e9f82945-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq\" (UID: \"683bce1b-f61f-428b-ad81-fae3e9f82945\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" Jul 16 12:14:41.855612 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:41.855444 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/683bce1b-f61f-428b-ad81-fae3e9f82945-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq\" (UID: \"683bce1b-f61f-428b-ad81-fae3e9f82945\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" Jul 16 12:14:41.855612 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:41.855507 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssc4x\" (UniqueName: \"kubernetes.io/projected/683bce1b-f61f-428b-ad81-fae3e9f82945-kube-api-access-ssc4x\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq\" (UID: \"683bce1b-f61f-428b-ad81-fae3e9f82945\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" Jul 16 12:14:41.855763 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:41.855741 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/683bce1b-f61f-428b-ad81-fae3e9f82945-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq\" (UID: \"683bce1b-f61f-428b-ad81-fae3e9f82945\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" Jul 16 12:14:41.855819 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:41.855801 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/683bce1b-f61f-428b-ad81-fae3e9f82945-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq\" (UID: \"683bce1b-f61f-428b-ad81-fae3e9f82945\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" Jul 16 12:14:41.872345 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:41.872310 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssc4x\" (UniqueName: \"kubernetes.io/projected/683bce1b-f61f-428b-ad81-fae3e9f82945-kube-api-access-ssc4x\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq\" (UID: \"683bce1b-f61f-428b-ad81-fae3e9f82945\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" Jul 16 12:14:42.038005 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:42.037970 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" Jul 16 12:14:42.177219 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:42.177187 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:42.202029 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:42.202002 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq"] Jul 16 12:14:42.203583 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:14:42.203549 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod683bce1b_f61f_428b_ad81_fae3e9f82945.slice/crio-39f427033a758b8be37d2e7d8bd114700571ad877450ea44841f842e8c06dde2 WatchSource:0}: Error finding container 39f427033a758b8be37d2e7d8bd114700571ad877450ea44841f842e8c06dde2: Status 404 returned error can't find the container with id 39f427033a758b8be37d2e7d8bd114700571ad877450ea44841f842e8c06dde2 Jul 16 12:14:42.262666 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:42.262635 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" event={"ID":"683bce1b-f61f-428b-ad81-fae3e9f82945","Type":"ContainerStarted","Data":"39f427033a758b8be37d2e7d8bd114700571ad877450ea44841f842e8c06dde2"} Jul 16 12:14:43.268937 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:43.268850 2578 generic.go:356] "Generic (PLEG): container finished" podID="683bce1b-f61f-428b-ad81-fae3e9f82945" containerID="652c5044e8337124f35ce4aac71a22b73dcf747e14184e7a4889db865cd93634" exitCode=0 Jul 16 12:14:43.268937 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:43.268905 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" event={"ID":"683bce1b-f61f-428b-ad81-fae3e9f82945","Type":"ContainerDied","Data":"652c5044e8337124f35ce4aac71a22b73dcf747e14184e7a4889db865cd93634"} Jul 16 12:14:44.093834 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:44.093800 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:44.244291 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:44.244252 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:14:46.285383 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:46.285348 2578 generic.go:356] "Generic (PLEG): container finished" podID="683bce1b-f61f-428b-ad81-fae3e9f82945" containerID="e4132d26e003a3ab0a1c0f97b461dd7a7c49a90ec077c0d0dd1cddb500b02f2c" exitCode=0 Jul 16 12:14:46.285817 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:46.285437 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" event={"ID":"683bce1b-f61f-428b-ad81-fae3e9f82945","Type":"ContainerDied","Data":"e4132d26e003a3ab0a1c0f97b461dd7a7c49a90ec077c0d0dd1cddb500b02f2c"} Jul 16 12:14:47.292258 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:47.292225 2578 generic.go:356] "Generic (PLEG): container finished" podID="683bce1b-f61f-428b-ad81-fae3e9f82945" containerID="c0e52339cc94bdf9b8ad3be7dfee68ac1cfb720ccb3b29abfb6a9bb531f72c33" exitCode=0 Jul 16 12:14:47.292661 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:47.292315 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" event={"ID":"683bce1b-f61f-428b-ad81-fae3e9f82945","Type":"ContainerDied","Data":"c0e52339cc94bdf9b8ad3be7dfee68ac1cfb720ccb3b29abfb6a9bb531f72c33"} Jul 16 12:14:48.425082 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:48.425055 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" Jul 16 12:14:48.513265 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:48.513227 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssc4x\" (UniqueName: \"kubernetes.io/projected/683bce1b-f61f-428b-ad81-fae3e9f82945-kube-api-access-ssc4x\") pod \"683bce1b-f61f-428b-ad81-fae3e9f82945\" (UID: \"683bce1b-f61f-428b-ad81-fae3e9f82945\") " Jul 16 12:14:48.513454 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:48.513326 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/683bce1b-f61f-428b-ad81-fae3e9f82945-bundle\") pod \"683bce1b-f61f-428b-ad81-fae3e9f82945\" (UID: \"683bce1b-f61f-428b-ad81-fae3e9f82945\") " Jul 16 12:14:48.513454 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:48.513374 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/683bce1b-f61f-428b-ad81-fae3e9f82945-util\") pod \"683bce1b-f61f-428b-ad81-fae3e9f82945\" (UID: \"683bce1b-f61f-428b-ad81-fae3e9f82945\") " Jul 16 12:14:48.514099 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:48.514070 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/683bce1b-f61f-428b-ad81-fae3e9f82945-bundle" (OuterVolumeSpecName: "bundle") pod "683bce1b-f61f-428b-ad81-fae3e9f82945" (UID: "683bce1b-f61f-428b-ad81-fae3e9f82945"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:14:48.515400 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:48.515373 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/683bce1b-f61f-428b-ad81-fae3e9f82945-kube-api-access-ssc4x" (OuterVolumeSpecName: "kube-api-access-ssc4x") pod "683bce1b-f61f-428b-ad81-fae3e9f82945" (UID: "683bce1b-f61f-428b-ad81-fae3e9f82945"). InnerVolumeSpecName "kube-api-access-ssc4x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:14:48.518584 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:48.518554 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/683bce1b-f61f-428b-ad81-fae3e9f82945-util" (OuterVolumeSpecName: "util") pod "683bce1b-f61f-428b-ad81-fae3e9f82945" (UID: "683bce1b-f61f-428b-ad81-fae3e9f82945"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:14:48.614558 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:48.614454 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/683bce1b-f61f-428b-ad81-fae3e9f82945-bundle\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:14:48.614558 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:48.614496 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/683bce1b-f61f-428b-ad81-fae3e9f82945-util\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:14:48.614558 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:48.614509 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssc4x\" (UniqueName: \"kubernetes.io/projected/683bce1b-f61f-428b-ad81-fae3e9f82945-kube-api-access-ssc4x\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:14:49.304541 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:49.304509 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" Jul 16 12:14:49.304708 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:49.304513 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxs2cq" event={"ID":"683bce1b-f61f-428b-ad81-fae3e9f82945","Type":"ContainerDied","Data":"39f427033a758b8be37d2e7d8bd114700571ad877450ea44841f842e8c06dde2"} Jul 16 12:14:49.304708 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:14:49.304628 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39f427033a758b8be37d2e7d8bd114700571ad877450ea44841f842e8c06dde2" Jul 16 12:15:00.117765 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.117665 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:00.214051 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.214010 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:00.745912 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.745876 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:00.903032 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.902999 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs"] Jul 16 12:15:00.903334 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.903323 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="683bce1b-f61f-428b-ad81-fae3e9f82945" containerName="util" Jul 16 12:15:00.903449 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.903336 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="683bce1b-f61f-428b-ad81-fae3e9f82945" containerName="util" Jul 16 12:15:00.903449 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.903343 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="683bce1b-f61f-428b-ad81-fae3e9f82945" containerName="pull" Jul 16 12:15:00.903449 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.903348 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="683bce1b-f61f-428b-ad81-fae3e9f82945" containerName="pull" Jul 16 12:15:00.903449 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.903359 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="683bce1b-f61f-428b-ad81-fae3e9f82945" containerName="extract" Jul 16 12:15:00.903449 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.903364 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="683bce1b-f61f-428b-ad81-fae3e9f82945" containerName="extract" Jul 16 12:15:00.903449 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.903438 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="683bce1b-f61f-428b-ad81-fae3e9f82945" containerName="extract" Jul 16 12:15:00.910291 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.910275 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" Jul 16 12:15:00.915773 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.915753 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 16 12:15:00.915908 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.915769 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-q44qf\"" Jul 16 12:15:00.915970 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.915938 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 16 12:15:00.916011 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.915979 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 16 12:15:00.918993 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.918967 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:00.933314 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:00.933294 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs"] Jul 16 12:15:01.012489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.012392 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b19e6162-bd6b-4d2b-8917-448e7709a1af-manager-config\") pod \"lws-controller-manager-69c8d477dd-rcnbs\" (UID: \"b19e6162-bd6b-4d2b-8917-448e7709a1af\") " pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" Jul 16 12:15:01.012489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.012470 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b19e6162-bd6b-4d2b-8917-448e7709a1af-metrics-cert\") pod \"lws-controller-manager-69c8d477dd-rcnbs\" (UID: \"b19e6162-bd6b-4d2b-8917-448e7709a1af\") " pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" Jul 16 12:15:01.012687 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.012590 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sft2\" (UniqueName: \"kubernetes.io/projected/b19e6162-bd6b-4d2b-8917-448e7709a1af-kube-api-access-4sft2\") pod \"lws-controller-manager-69c8d477dd-rcnbs\" (UID: \"b19e6162-bd6b-4d2b-8917-448e7709a1af\") " pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" Jul 16 12:15:01.012687 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.012636 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b19e6162-bd6b-4d2b-8917-448e7709a1af-cert\") pod \"lws-controller-manager-69c8d477dd-rcnbs\" (UID: \"b19e6162-bd6b-4d2b-8917-448e7709a1af\") " pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" Jul 16 12:15:01.113157 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.113128 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sft2\" (UniqueName: \"kubernetes.io/projected/b19e6162-bd6b-4d2b-8917-448e7709a1af-kube-api-access-4sft2\") pod \"lws-controller-manager-69c8d477dd-rcnbs\" (UID: \"b19e6162-bd6b-4d2b-8917-448e7709a1af\") " pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" Jul 16 12:15:01.113329 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.113171 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b19e6162-bd6b-4d2b-8917-448e7709a1af-cert\") pod \"lws-controller-manager-69c8d477dd-rcnbs\" (UID: \"b19e6162-bd6b-4d2b-8917-448e7709a1af\") " pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" Jul 16 12:15:01.113329 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.113207 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b19e6162-bd6b-4d2b-8917-448e7709a1af-manager-config\") pod \"lws-controller-manager-69c8d477dd-rcnbs\" (UID: \"b19e6162-bd6b-4d2b-8917-448e7709a1af\") " pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" Jul 16 12:15:01.113329 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.113235 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b19e6162-bd6b-4d2b-8917-448e7709a1af-metrics-cert\") pod \"lws-controller-manager-69c8d477dd-rcnbs\" (UID: \"b19e6162-bd6b-4d2b-8917-448e7709a1af\") " pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" Jul 16 12:15:01.113892 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.113867 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b19e6162-bd6b-4d2b-8917-448e7709a1af-manager-config\") pod \"lws-controller-manager-69c8d477dd-rcnbs\" (UID: \"b19e6162-bd6b-4d2b-8917-448e7709a1af\") " pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" Jul 16 12:15:01.115731 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.115708 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b19e6162-bd6b-4d2b-8917-448e7709a1af-cert\") pod \"lws-controller-manager-69c8d477dd-rcnbs\" (UID: \"b19e6162-bd6b-4d2b-8917-448e7709a1af\") " pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" Jul 16 12:15:01.115972 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.115953 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b19e6162-bd6b-4d2b-8917-448e7709a1af-metrics-cert\") pod \"lws-controller-manager-69c8d477dd-rcnbs\" (UID: \"b19e6162-bd6b-4d2b-8917-448e7709a1af\") " pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" Jul 16 12:15:01.127677 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.127651 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sft2\" (UniqueName: \"kubernetes.io/projected/b19e6162-bd6b-4d2b-8917-448e7709a1af-kube-api-access-4sft2\") pod \"lws-controller-manager-69c8d477dd-rcnbs\" (UID: \"b19e6162-bd6b-4d2b-8917-448e7709a1af\") " pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" Jul 16 12:15:01.210160 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.210129 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:01.220592 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.220571 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" Jul 16 12:15:01.333118 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.333085 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:01.366280 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.366252 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm"] Jul 16 12:15:01.371260 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.371241 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.373783 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.373763 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs"] Jul 16 12:15:01.375287 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.375265 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 12:15:01.375395 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.375360 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 12:15:01.375478 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.375392 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 12:15:01.375535 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:15:01.375515 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb19e6162_bd6b_4d2b_8917_448e7709a1af.slice/crio-ef14535c955bb85a9ed28e61a8d77e597b9dda78e3d266d882db2ca0c2c3078d WatchSource:0}: Error finding container ef14535c955bb85a9ed28e61a8d77e597b9dda78e3d266d882db2ca0c2c3078d: Status 404 returned error can't find the container with id ef14535c955bb85a9ed28e61a8d77e597b9dda78e3d266d882db2ca0c2c3078d Jul 16 12:15:01.375594 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.375533 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 12:15:01.376010 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.375990 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-m9js4\"" Jul 16 12:15:01.403852 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.403830 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm"] Jul 16 12:15:01.517009 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.516970 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff4zw\" (UniqueName: \"kubernetes.io/projected/92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0-kube-api-access-ff4zw\") pod \"opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm\" (UID: \"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.517178 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.517040 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm\" (UID: \"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.517178 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.517066 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0-tmp\") pod \"opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm\" (UID: \"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.517178 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.517089 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm\" (UID: \"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.517325 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.517171 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0-manifests\") pod \"opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm\" (UID: \"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.618462 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.618400 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm\" (UID: \"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.618659 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.618472 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0-tmp\") pod \"opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm\" (UID: \"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.618659 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.618513 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm\" (UID: \"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.618659 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.618564 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0-manifests\") pod \"opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm\" (UID: \"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.618659 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.618632 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff4zw\" (UniqueName: \"kubernetes.io/projected/92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0-kube-api-access-ff4zw\") pod \"opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm\" (UID: \"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.619021 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.618998 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0-manifests\") pod \"opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm\" (UID: \"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.620853 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.620830 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0-tmp\") pod \"opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm\" (UID: \"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.620934 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.620923 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm\" (UID: \"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.620985 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.620967 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm\" (UID: \"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.624965 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.624945 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:01.629712 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.629692 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff4zw\" (UniqueName: \"kubernetes.io/projected/92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0-kube-api-access-ff4zw\") pod \"opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm\" (UID: \"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.681938 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.681899 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:01.778393 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.778346 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:01.858363 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.858340 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm"] Jul 16 12:15:01.859940 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:15:01.859917 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92cb28e2_5b64_4acd_b4b5_e2fbfd2ec5e0.slice/crio-2a897c01720d6dc4bf0371c9214bd7cb368602a3cdf1a5270a8bc8f089ed2c73 WatchSource:0}: Error finding container 2a897c01720d6dc4bf0371c9214bd7cb368602a3cdf1a5270a8bc8f089ed2c73: Status 404 returned error can't find the container with id 2a897c01720d6dc4bf0371c9214bd7cb368602a3cdf1a5270a8bc8f089ed2c73 Jul 16 12:15:01.903101 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:01.903009 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:02.050806 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:02.050774 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:02.280908 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:02.280882 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:02.367776 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:02.367733 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" event={"ID":"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0","Type":"ContainerStarted","Data":"2a897c01720d6dc4bf0371c9214bd7cb368602a3cdf1a5270a8bc8f089ed2c73"} Jul 16 12:15:02.368855 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:02.368832 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" event={"ID":"b19e6162-bd6b-4d2b-8917-448e7709a1af","Type":"ContainerStarted","Data":"ef14535c955bb85a9ed28e61a8d77e597b9dda78e3d266d882db2ca0c2c3078d"} Jul 16 12:15:07.227659 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:07.227572 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:07.397123 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:07.397075 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" event={"ID":"b19e6162-bd6b-4d2b-8917-448e7709a1af","Type":"ContainerStarted","Data":"a83197960b082485643551ba7e4cf071d535005b43bfad2004303172e65b875a"} Jul 16 12:15:07.397300 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:07.397178 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" Jul 16 12:15:07.398676 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:07.398653 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" event={"ID":"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0","Type":"ContainerStarted","Data":"4fedd01213e118ec026793527078c7cd1e944e1ced6d548d61f01f0f50f53c79"} Jul 16 12:15:07.461946 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:07.461887 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" podStartSLOduration=1.978241953 podStartE2EDuration="7.461872036s" totalImagesPullingTime="5.483630083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:15:07.46089798 +0000 UTC m=+457.224267445" watchObservedRunningTime="2026-07-16 12:15:07.461872036 +0000 UTC m=+457.225241499" Jul 16 12:15:08.405200 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:08.405154 2578 generic.go:356] "Generic (PLEG): container finished" podID="92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0" containerID="4fedd01213e118ec026793527078c7cd1e944e1ced6d548d61f01f0f50f53c79" exitCode=0 Jul 16 12:15:08.405606 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:08.405236 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" event={"ID":"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0","Type":"ContainerDied","Data":"4fedd01213e118ec026793527078c7cd1e944e1ced6d548d61f01f0f50f53c79"} Jul 16 12:15:09.411294 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:09.411256 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" event={"ID":"92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0","Type":"ContainerStarted","Data":"a9151812720af66ffd9c4f7d34d425b90b28a1c506531c5795b529bd774ffc2b"} Jul 16 12:15:09.411746 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:09.411445 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:09.445226 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:09.445180 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" podStartSLOduration=1.907430566 podStartE2EDuration="8.445166256s" totalImagesPullingTime="5.455581995s" totalInitContainerRuntime="1.082153695s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:15:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:15:09.444871802 +0000 UTC m=+459.208241264" watchObservedRunningTime="2026-07-16 12:15:09.445166256 +0000 UTC m=+459.208535719" Jul 16 12:15:18.408188 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:18.408154 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-69c8d477dd-rcnbs" Jul 16 12:15:20.417994 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:20.417961 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm" Jul 16 12:15:22.878536 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:22.878502 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:23.239709 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:23.239616 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:24.149376 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:24.149342 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:24.279665 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:24.279616 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:24.396868 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:24.396836 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:24.518981 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:24.518889 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:25.260908 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:25.260877 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:25.628303 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:25.628268 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:29.688045 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:29.688008 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:29.785141 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:29.785103 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:31.686059 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:31.686024 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:31.918226 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:31.918192 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:31.997258 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:31.997169 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:33.434575 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.434547 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz"] Jul 16 12:15:33.437903 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.437886 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.440992 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.440969 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 16 12:15:33.441121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.441063 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 16 12:15:33.441121 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.441063 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 16 12:15:33.442243 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.441567 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 12:15:33.442243 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.441854 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-j6lrv\"" Jul 16 12:15:33.458779 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.458746 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz"] Jul 16 12:15:33.501698 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.501662 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/956ffded-32d6-416b-baca-9de1b5cc5747-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.501870 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.501706 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/956ffded-32d6-416b-baca-9de1b5cc5747-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.501870 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.501730 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/956ffded-32d6-416b-baca-9de1b5cc5747-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.501870 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.501797 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/956ffded-32d6-416b-baca-9de1b5cc5747-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.501870 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.501828 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wttqn\" (UniqueName: \"kubernetes.io/projected/956ffded-32d6-416b-baca-9de1b5cc5747-kube-api-access-wttqn\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.501870 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.501851 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/956ffded-32d6-416b-baca-9de1b5cc5747-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.502080 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.501895 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/956ffded-32d6-416b-baca-9de1b5cc5747-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.545335 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.545294 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:33.602614 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.602586 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/956ffded-32d6-416b-baca-9de1b5cc5747-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.602777 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.602665 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/956ffded-32d6-416b-baca-9de1b5cc5747-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.602777 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.602703 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/956ffded-32d6-416b-baca-9de1b5cc5747-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.602777 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.602732 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/956ffded-32d6-416b-baca-9de1b5cc5747-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.602777 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.602764 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/956ffded-32d6-416b-baca-9de1b5cc5747-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.602927 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.602784 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wttqn\" (UniqueName: \"kubernetes.io/projected/956ffded-32d6-416b-baca-9de1b5cc5747-kube-api-access-wttqn\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.602927 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.602810 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/956ffded-32d6-416b-baca-9de1b5cc5747-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.603402 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.603375 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/956ffded-32d6-416b-baca-9de1b5cc5747-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.604913 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.604884 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/956ffded-32d6-416b-baca-9de1b5cc5747-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.605085 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.605066 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/956ffded-32d6-416b-baca-9de1b5cc5747-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.605500 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.605473 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/956ffded-32d6-416b-baca-9de1b5cc5747-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.605578 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.605472 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/956ffded-32d6-416b-baca-9de1b5cc5747-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.612843 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.612823 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wttqn\" (UniqueName: \"kubernetes.io/projected/956ffded-32d6-416b-baca-9de1b5cc5747-kube-api-access-wttqn\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.617077 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.617058 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/956ffded-32d6-416b-baca-9de1b5cc5747-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6lsdz\" (UID: \"956ffded-32d6-416b-baca-9de1b5cc5747\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.752264 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.752171 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:33.911618 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:33.911592 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz"] Jul 16 12:15:33.913828 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:15:33.913793 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod956ffded_32d6_416b_baca_9de1b5cc5747.slice/crio-84b1ed2367b318565119f6dfdb7282e4a846ccda01045316909ca0ad96bea969 WatchSource:0}: Error finding container 84b1ed2367b318565119f6dfdb7282e4a846ccda01045316909ca0ad96bea969: Status 404 returned error can't find the container with id 84b1ed2367b318565119f6dfdb7282e4a846ccda01045316909ca0ad96bea969 Jul 16 12:15:34.530770 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:34.530732 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" event={"ID":"956ffded-32d6-416b-baca-9de1b5cc5747","Type":"ContainerStarted","Data":"84b1ed2367b318565119f6dfdb7282e4a846ccda01045316909ca0ad96bea969"} Jul 16 12:15:36.015736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:36.015694 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:15:36.016016 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:36.015763 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:15:36.354157 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:36.354120 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:36.464241 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:36.464204 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:36.544579 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:36.544542 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" event={"ID":"956ffded-32d6-416b-baca-9de1b5cc5747","Type":"ContainerStarted","Data":"01e59e3f643680521eec42882150aec6a73aa95f9e73ae93536ccf7dafba843d"} Jul 16 12:15:36.544746 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:36.544646 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:36.554084 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:36.554055 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:36.573023 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:36.572970 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" podStartSLOduration=1.473415113 podStartE2EDuration="3.572955662s" totalImagesPullingTime="2.099540549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:15:36.570711408 +0000 UTC m=+486.334080904" watchObservedRunningTime="2026-07-16 12:15:36.572955662 +0000 UTC m=+486.336325124" Jul 16 12:15:36.895134 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:36.895096 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:37.096040 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:37.096011 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:37.224006 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:37.223913 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:37.552050 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:37.552023 2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-6lsdz container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 12:15:37.552246 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:37.552070 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" podUID="956ffded-32d6-416b-baca-9de1b5cc5747" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 12:15:38.117237 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:38.117200 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:38.306780 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:38.306746 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:40.096604 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:40.096561 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:40.451730 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:40.451643 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:40.551651 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:40.551622 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6lsdz" Jul 16 12:15:40.575455 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:40.575395 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:41.029010 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:41.028969 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:46.974377 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:46.974336 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:15:55.280771 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:15:55.280728 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:03.934278 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:03.934241 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:24.651220 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:24.651118 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:24.740075 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:24.740048 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:24.892962 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:24.892929 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:25.254116 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:25.254078 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:25.363336 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:25.363300 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:25.656337 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:25.656297 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:25.860118 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:25.860088 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:37.399717 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:37.399685 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:37.502808 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:37.502778 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:41.037626 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.037589 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k"] Jul 16 12:16:41.040698 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.040675 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" Jul 16 12:16:41.045297 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.045277 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 12:16:41.045396 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.045362 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 12:16:41.046581 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.046564 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tf89p\"" Jul 16 12:16:41.062522 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.062505 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k"] Jul 16 12:16:41.095210 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.095189 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1323e7ab-7964-460c-a2d0-77fbced802b5-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k\" (UID: \"1323e7ab-7964-460c-a2d0-77fbced802b5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" Jul 16 12:16:41.095321 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.095237 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1323e7ab-7964-460c-a2d0-77fbced802b5-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k\" (UID: \"1323e7ab-7964-460c-a2d0-77fbced802b5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" Jul 16 12:16:41.095388 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.095341 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xltn\" (UniqueName: \"kubernetes.io/projected/1323e7ab-7964-460c-a2d0-77fbced802b5-kube-api-access-8xltn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k\" (UID: \"1323e7ab-7964-460c-a2d0-77fbced802b5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" Jul 16 12:16:41.195947 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.195917 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xltn\" (UniqueName: \"kubernetes.io/projected/1323e7ab-7964-460c-a2d0-77fbced802b5-kube-api-access-8xltn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k\" (UID: \"1323e7ab-7964-460c-a2d0-77fbced802b5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" Jul 16 12:16:41.196061 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.195973 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1323e7ab-7964-460c-a2d0-77fbced802b5-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k\" (UID: \"1323e7ab-7964-460c-a2d0-77fbced802b5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" Jul 16 12:16:41.196061 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.196002 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1323e7ab-7964-460c-a2d0-77fbced802b5-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k\" (UID: \"1323e7ab-7964-460c-a2d0-77fbced802b5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" Jul 16 12:16:41.196357 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.196340 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1323e7ab-7964-460c-a2d0-77fbced802b5-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k\" (UID: \"1323e7ab-7964-460c-a2d0-77fbced802b5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" Jul 16 12:16:41.196585 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.196366 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1323e7ab-7964-460c-a2d0-77fbced802b5-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k\" (UID: \"1323e7ab-7964-460c-a2d0-77fbced802b5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" Jul 16 12:16:41.208360 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.208338 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xltn\" (UniqueName: \"kubernetes.io/projected/1323e7ab-7964-460c-a2d0-77fbced802b5-kube-api-access-8xltn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k\" (UID: \"1323e7ab-7964-460c-a2d0-77fbced802b5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" Jul 16 12:16:41.349993 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.349915 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" Jul 16 12:16:41.492974 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.492951 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k"] Jul 16 12:16:41.494670 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:16:41.494638 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1323e7ab_7964_460c_a2d0_77fbced802b5.slice/crio-02afd2b595cecd8206f326f6e775c70ae0fa946329009bba7bd92cd288b19e9d WatchSource:0}: Error finding container 02afd2b595cecd8206f326f6e775c70ae0fa946329009bba7bd92cd288b19e9d: Status 404 returned error can't find the container with id 02afd2b595cecd8206f326f6e775c70ae0fa946329009bba7bd92cd288b19e9d Jul 16 12:16:41.656087 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.656024 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s"] Jul 16 12:16:41.658606 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.658591 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" Jul 16 12:16:41.678441 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.678399 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s"] Jul 16 12:16:41.802381 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.802342 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5bf6abd-db25-4f50-b5a2-49def53cb4af-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s\" (UID: \"f5bf6abd-db25-4f50-b5a2-49def53cb4af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" Jul 16 12:16:41.802643 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.802448 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5bf6abd-db25-4f50-b5a2-49def53cb4af-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s\" (UID: \"f5bf6abd-db25-4f50-b5a2-49def53cb4af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" Jul 16 12:16:41.802643 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.802476 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcthp\" (UniqueName: \"kubernetes.io/projected/f5bf6abd-db25-4f50-b5a2-49def53cb4af-kube-api-access-xcthp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s\" (UID: \"f5bf6abd-db25-4f50-b5a2-49def53cb4af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" Jul 16 12:16:41.861150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.861116 2578 generic.go:356] "Generic (PLEG): container finished" podID="1323e7ab-7964-460c-a2d0-77fbced802b5" containerID="2cf881a083329ec58cd7a3c3e9d851f08c76a2c4adaf474c72a865a76db96ac0" exitCode=0 Jul 16 12:16:41.861294 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.861157 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" event={"ID":"1323e7ab-7964-460c-a2d0-77fbced802b5","Type":"ContainerDied","Data":"2cf881a083329ec58cd7a3c3e9d851f08c76a2c4adaf474c72a865a76db96ac0"} Jul 16 12:16:41.861294 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.861202 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" event={"ID":"1323e7ab-7964-460c-a2d0-77fbced802b5","Type":"ContainerStarted","Data":"02afd2b595cecd8206f326f6e775c70ae0fa946329009bba7bd92cd288b19e9d"} Jul 16 12:16:41.903294 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.903261 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5bf6abd-db25-4f50-b5a2-49def53cb4af-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s\" (UID: \"f5bf6abd-db25-4f50-b5a2-49def53cb4af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" Jul 16 12:16:41.903453 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.903299 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcthp\" (UniqueName: \"kubernetes.io/projected/f5bf6abd-db25-4f50-b5a2-49def53cb4af-kube-api-access-xcthp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s\" (UID: \"f5bf6abd-db25-4f50-b5a2-49def53cb4af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" Jul 16 12:16:41.903453 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.903387 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5bf6abd-db25-4f50-b5a2-49def53cb4af-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s\" (UID: \"f5bf6abd-db25-4f50-b5a2-49def53cb4af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" Jul 16 12:16:41.903662 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.903624 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5bf6abd-db25-4f50-b5a2-49def53cb4af-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s\" (UID: \"f5bf6abd-db25-4f50-b5a2-49def53cb4af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" Jul 16 12:16:41.903738 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.903722 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5bf6abd-db25-4f50-b5a2-49def53cb4af-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s\" (UID: \"f5bf6abd-db25-4f50-b5a2-49def53cb4af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" Jul 16 12:16:41.917888 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.917834 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcthp\" (UniqueName: \"kubernetes.io/projected/f5bf6abd-db25-4f50-b5a2-49def53cb4af-kube-api-access-xcthp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s\" (UID: \"f5bf6abd-db25-4f50-b5a2-49def53cb4af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" Jul 16 12:16:41.979010 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:41.978973 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" Jul 16 12:16:42.063576 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.063548 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k"] Jul 16 12:16:42.066494 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.066473 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" Jul 16 12:16:42.091768 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.091734 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k"] Jul 16 12:16:42.120999 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.120976 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s"] Jul 16 12:16:42.122553 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:16:42.122531 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5bf6abd_db25_4f50_b5a2_49def53cb4af.slice/crio-ca146f2e315087cfa46244333897c896011582ddcc18fee61381785868890cb7 WatchSource:0}: Error finding container ca146f2e315087cfa46244333897c896011582ddcc18fee61381785868890cb7: Status 404 returned error can't find the container with id ca146f2e315087cfa46244333897c896011582ddcc18fee61381785868890cb7 Jul 16 12:16:42.206054 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.205972 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e13484fb-1430-473c-9b88-0d0fbd8e76cc-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k\" (UID: \"e13484fb-1430-473c-9b88-0d0fbd8e76cc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" Jul 16 12:16:42.206054 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.206038 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e13484fb-1430-473c-9b88-0d0fbd8e76cc-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k\" (UID: \"e13484fb-1430-473c-9b88-0d0fbd8e76cc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" Jul 16 12:16:42.206243 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.206076 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j52dc\" (UniqueName: \"kubernetes.io/projected/e13484fb-1430-473c-9b88-0d0fbd8e76cc-kube-api-access-j52dc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k\" (UID: \"e13484fb-1430-473c-9b88-0d0fbd8e76cc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" Jul 16 12:16:42.307490 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.307460 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e13484fb-1430-473c-9b88-0d0fbd8e76cc-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k\" (UID: \"e13484fb-1430-473c-9b88-0d0fbd8e76cc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" Jul 16 12:16:42.307664 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.307504 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j52dc\" (UniqueName: \"kubernetes.io/projected/e13484fb-1430-473c-9b88-0d0fbd8e76cc-kube-api-access-j52dc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k\" (UID: \"e13484fb-1430-473c-9b88-0d0fbd8e76cc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" Jul 16 12:16:42.307664 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.307594 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e13484fb-1430-473c-9b88-0d0fbd8e76cc-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k\" (UID: \"e13484fb-1430-473c-9b88-0d0fbd8e76cc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" Jul 16 12:16:42.307861 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.307842 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e13484fb-1430-473c-9b88-0d0fbd8e76cc-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k\" (UID: \"e13484fb-1430-473c-9b88-0d0fbd8e76cc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" Jul 16 12:16:42.307923 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.307908 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e13484fb-1430-473c-9b88-0d0fbd8e76cc-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k\" (UID: \"e13484fb-1430-473c-9b88-0d0fbd8e76cc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" Jul 16 12:16:42.317181 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.317154 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j52dc\" (UniqueName: \"kubernetes.io/projected/e13484fb-1430-473c-9b88-0d0fbd8e76cc-kube-api-access-j52dc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k\" (UID: \"e13484fb-1430-473c-9b88-0d0fbd8e76cc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" Jul 16 12:16:42.379115 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.379076 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" Jul 16 12:16:42.416856 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.416818 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq"] Jul 16 12:16:42.419849 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.419828 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" Jul 16 12:16:42.431989 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.431952 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq"] Jul 16 12:16:42.509622 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.509593 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq\" (UID: \"c09bf55c-8a63-49eb-9f5a-d09ce780a38a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" Jul 16 12:16:42.509789 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.509633 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq\" (UID: \"c09bf55c-8a63-49eb-9f5a-d09ce780a38a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" Jul 16 12:16:42.509789 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.509665 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtn8r\" (UniqueName: \"kubernetes.io/projected/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-kube-api-access-gtn8r\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq\" (UID: \"c09bf55c-8a63-49eb-9f5a-d09ce780a38a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" Jul 16 12:16:42.516934 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.516908 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k"] Jul 16 12:16:42.518539 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:16:42.518513 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode13484fb_1430_473c_9b88_0d0fbd8e76cc.slice/crio-53c127c7b7596ba5ad73ff8670f96b7484179e72c09b2ef1261556c774be79a4 WatchSource:0}: Error finding container 53c127c7b7596ba5ad73ff8670f96b7484179e72c09b2ef1261556c774be79a4: Status 404 returned error can't find the container with id 53c127c7b7596ba5ad73ff8670f96b7484179e72c09b2ef1261556c774be79a4 Jul 16 12:16:42.610343 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.610315 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq\" (UID: \"c09bf55c-8a63-49eb-9f5a-d09ce780a38a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" Jul 16 12:16:42.610472 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.610377 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq\" (UID: \"c09bf55c-8a63-49eb-9f5a-d09ce780a38a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" Jul 16 12:16:42.610472 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.610445 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtn8r\" (UniqueName: \"kubernetes.io/projected/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-kube-api-access-gtn8r\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq\" (UID: \"c09bf55c-8a63-49eb-9f5a-d09ce780a38a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" Jul 16 12:16:42.610745 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.610724 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq\" (UID: \"c09bf55c-8a63-49eb-9f5a-d09ce780a38a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" Jul 16 12:16:42.610783 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.610732 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq\" (UID: \"c09bf55c-8a63-49eb-9f5a-d09ce780a38a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" Jul 16 12:16:42.620236 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.620210 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtn8r\" (UniqueName: \"kubernetes.io/projected/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-kube-api-access-gtn8r\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq\" (UID: \"c09bf55c-8a63-49eb-9f5a-d09ce780a38a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" Jul 16 12:16:42.731768 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.731693 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" Jul 16 12:16:42.867066 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.867030 2578 generic.go:356] "Generic (PLEG): container finished" podID="e13484fb-1430-473c-9b88-0d0fbd8e76cc" containerID="2094e93293fcfa6f1875444141b7116f09c2ec1cbe4a187dc705b6ebd7adbbc5" exitCode=0 Jul 16 12:16:42.867228 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.867114 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" event={"ID":"e13484fb-1430-473c-9b88-0d0fbd8e76cc","Type":"ContainerDied","Data":"2094e93293fcfa6f1875444141b7116f09c2ec1cbe4a187dc705b6ebd7adbbc5"} Jul 16 12:16:42.867228 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.867161 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" event={"ID":"e13484fb-1430-473c-9b88-0d0fbd8e76cc","Type":"ContainerStarted","Data":"53c127c7b7596ba5ad73ff8670f96b7484179e72c09b2ef1261556c774be79a4"} Jul 16 12:16:42.869180 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.869154 2578 generic.go:356] "Generic (PLEG): container finished" podID="f5bf6abd-db25-4f50-b5a2-49def53cb4af" containerID="fb433064bc2b9353919715d0e184dc4f0faa4f840794348db5ab603d03cf9094" exitCode=0 Jul 16 12:16:42.869296 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.869187 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" event={"ID":"f5bf6abd-db25-4f50-b5a2-49def53cb4af","Type":"ContainerDied","Data":"fb433064bc2b9353919715d0e184dc4f0faa4f840794348db5ab603d03cf9094"} Jul 16 12:16:42.869296 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.869220 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" event={"ID":"f5bf6abd-db25-4f50-b5a2-49def53cb4af","Type":"ContainerStarted","Data":"ca146f2e315087cfa46244333897c896011582ddcc18fee61381785868890cb7"} Jul 16 12:16:42.871492 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:42.871471 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq"] Jul 16 12:16:42.873483 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:16:42.873451 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc09bf55c_8a63_49eb_9f5a_d09ce780a38a.slice/crio-88cc9d29ea4629b9c0588144f6f744c9b028db5e5521f11a3295184d08643067 WatchSource:0}: Error finding container 88cc9d29ea4629b9c0588144f6f744c9b028db5e5521f11a3295184d08643067: Status 404 returned error can't find the container with id 88cc9d29ea4629b9c0588144f6f744c9b028db5e5521f11a3295184d08643067 Jul 16 12:16:43.877226 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:43.877192 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" event={"ID":"f5bf6abd-db25-4f50-b5a2-49def53cb4af","Type":"ContainerStarted","Data":"3c4251489e3c4e4510c68bcfa9fa13d9c206a69ff8ab72cfd0d43f320e11bb68"} Jul 16 12:16:43.879308 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:43.879282 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" event={"ID":"1323e7ab-7964-460c-a2d0-77fbced802b5","Type":"ContainerStarted","Data":"c86fcd739c8f6f585444e6600b772158f1553292bfef4f10af8ef4f42903714c"} Jul 16 12:16:43.880982 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:43.880956 2578 generic.go:356] "Generic (PLEG): container finished" podID="c09bf55c-8a63-49eb-9f5a-d09ce780a38a" containerID="91a94687223090fcfed026fbd392a8ad411d40ec6cb2672c3bd338a0fb81f865" exitCode=0 Jul 16 12:16:43.881085 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:43.881047 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" event={"ID":"c09bf55c-8a63-49eb-9f5a-d09ce780a38a","Type":"ContainerDied","Data":"91a94687223090fcfed026fbd392a8ad411d40ec6cb2672c3bd338a0fb81f865"} Jul 16 12:16:43.881136 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:43.881093 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" event={"ID":"c09bf55c-8a63-49eb-9f5a-d09ce780a38a","Type":"ContainerStarted","Data":"88cc9d29ea4629b9c0588144f6f744c9b028db5e5521f11a3295184d08643067"} Jul 16 12:16:44.888201 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:44.888164 2578 generic.go:356] "Generic (PLEG): container finished" podID="f5bf6abd-db25-4f50-b5a2-49def53cb4af" containerID="3c4251489e3c4e4510c68bcfa9fa13d9c206a69ff8ab72cfd0d43f320e11bb68" exitCode=0 Jul 16 12:16:44.888686 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:44.888245 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" event={"ID":"f5bf6abd-db25-4f50-b5a2-49def53cb4af","Type":"ContainerDied","Data":"3c4251489e3c4e4510c68bcfa9fa13d9c206a69ff8ab72cfd0d43f320e11bb68"} Jul 16 12:16:44.890218 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:44.890197 2578 generic.go:356] "Generic (PLEG): container finished" podID="1323e7ab-7964-460c-a2d0-77fbced802b5" containerID="c86fcd739c8f6f585444e6600b772158f1553292bfef4f10af8ef4f42903714c" exitCode=0 Jul 16 12:16:44.890345 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:44.890323 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" event={"ID":"1323e7ab-7964-460c-a2d0-77fbced802b5","Type":"ContainerDied","Data":"c86fcd739c8f6f585444e6600b772158f1553292bfef4f10af8ef4f42903714c"} Jul 16 12:16:45.897353 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:45.897320 2578 generic.go:356] "Generic (PLEG): container finished" podID="f5bf6abd-db25-4f50-b5a2-49def53cb4af" containerID="583c855d38136c8612f8b66c259b86b027b7fa63dac65e20bb711b0531a2023b" exitCode=0 Jul 16 12:16:45.897671 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:45.897396 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" event={"ID":"f5bf6abd-db25-4f50-b5a2-49def53cb4af","Type":"ContainerDied","Data":"583c855d38136c8612f8b66c259b86b027b7fa63dac65e20bb711b0531a2023b"} Jul 16 12:16:45.899253 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:45.899234 2578 generic.go:356] "Generic (PLEG): container finished" podID="1323e7ab-7964-460c-a2d0-77fbced802b5" containerID="1b6c0a0e3ed53560da80ec1f23b6d6509eec365be1bbb49ebc63543a5867249d" exitCode=0 Jul 16 12:16:45.899346 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:45.899307 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" event={"ID":"1323e7ab-7964-460c-a2d0-77fbced802b5","Type":"ContainerDied","Data":"1b6c0a0e3ed53560da80ec1f23b6d6509eec365be1bbb49ebc63543a5867249d"} Jul 16 12:16:45.900906 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:45.900885 2578 generic.go:356] "Generic (PLEG): container finished" podID="e13484fb-1430-473c-9b88-0d0fbd8e76cc" containerID="66300fd9c6fe1ac8e13d2be707e74f7096388e07cbd1af1ebf0dbe9ea15987fd" exitCode=0 Jul 16 12:16:45.900981 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:45.900945 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" event={"ID":"e13484fb-1430-473c-9b88-0d0fbd8e76cc","Type":"ContainerDied","Data":"66300fd9c6fe1ac8e13d2be707e74f7096388e07cbd1af1ebf0dbe9ea15987fd"} Jul 16 12:16:45.902762 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:45.902741 2578 generic.go:356] "Generic (PLEG): container finished" podID="c09bf55c-8a63-49eb-9f5a-d09ce780a38a" containerID="d22ae871dd3e5f7716644c23c411518095fd956f3d2a35217f87f15814a70b82" exitCode=0 Jul 16 12:16:45.902836 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:45.902798 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" event={"ID":"c09bf55c-8a63-49eb-9f5a-d09ce780a38a","Type":"ContainerDied","Data":"d22ae871dd3e5f7716644c23c411518095fd956f3d2a35217f87f15814a70b82"} Jul 16 12:16:46.913199 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:46.913169 2578 generic.go:356] "Generic (PLEG): container finished" podID="e13484fb-1430-473c-9b88-0d0fbd8e76cc" containerID="9d1f5c379ef5f2b8d169227ab08f9bf331f41a3fc9b5dcfc9a99cff7798765e0" exitCode=0 Jul 16 12:16:46.913611 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:46.913247 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" event={"ID":"e13484fb-1430-473c-9b88-0d0fbd8e76cc","Type":"ContainerDied","Data":"9d1f5c379ef5f2b8d169227ab08f9bf331f41a3fc9b5dcfc9a99cff7798765e0"} Jul 16 12:16:46.915014 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:46.914991 2578 generic.go:356] "Generic (PLEG): container finished" podID="c09bf55c-8a63-49eb-9f5a-d09ce780a38a" containerID="6bade82216bc1aabd5e4f5acd072e07f47bcc7048e15bae671a80e38da17750d" exitCode=0 Jul 16 12:16:46.915139 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:46.915068 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" event={"ID":"c09bf55c-8a63-49eb-9f5a-d09ce780a38a","Type":"ContainerDied","Data":"6bade82216bc1aabd5e4f5acd072e07f47bcc7048e15bae671a80e38da17750d"} Jul 16 12:16:47.048895 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.048864 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" Jul 16 12:16:47.078689 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.078669 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" Jul 16 12:16:47.150384 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.150362 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1323e7ab-7964-460c-a2d0-77fbced802b5-bundle\") pod \"1323e7ab-7964-460c-a2d0-77fbced802b5\" (UID: \"1323e7ab-7964-460c-a2d0-77fbced802b5\") " Jul 16 12:16:47.150512 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.150391 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcthp\" (UniqueName: \"kubernetes.io/projected/f5bf6abd-db25-4f50-b5a2-49def53cb4af-kube-api-access-xcthp\") pod \"f5bf6abd-db25-4f50-b5a2-49def53cb4af\" (UID: \"f5bf6abd-db25-4f50-b5a2-49def53cb4af\") " Jul 16 12:16:47.150512 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.150475 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5bf6abd-db25-4f50-b5a2-49def53cb4af-util\") pod \"f5bf6abd-db25-4f50-b5a2-49def53cb4af\" (UID: \"f5bf6abd-db25-4f50-b5a2-49def53cb4af\") " Jul 16 12:16:47.150666 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.150649 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5bf6abd-db25-4f50-b5a2-49def53cb4af-bundle\") pod \"f5bf6abd-db25-4f50-b5a2-49def53cb4af\" (UID: \"f5bf6abd-db25-4f50-b5a2-49def53cb4af\") " Jul 16 12:16:47.150716 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.150692 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xltn\" (UniqueName: \"kubernetes.io/projected/1323e7ab-7964-460c-a2d0-77fbced802b5-kube-api-access-8xltn\") pod \"1323e7ab-7964-460c-a2d0-77fbced802b5\" (UID: \"1323e7ab-7964-460c-a2d0-77fbced802b5\") " Jul 16 12:16:47.150761 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.150738 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1323e7ab-7964-460c-a2d0-77fbced802b5-util\") pod \"1323e7ab-7964-460c-a2d0-77fbced802b5\" (UID: \"1323e7ab-7964-460c-a2d0-77fbced802b5\") " Jul 16 12:16:47.151158 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.151130 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1323e7ab-7964-460c-a2d0-77fbced802b5-bundle" (OuterVolumeSpecName: "bundle") pod "1323e7ab-7964-460c-a2d0-77fbced802b5" (UID: "1323e7ab-7964-460c-a2d0-77fbced802b5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:16:47.151265 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.151242 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5bf6abd-db25-4f50-b5a2-49def53cb4af-bundle" (OuterVolumeSpecName: "bundle") pod "f5bf6abd-db25-4f50-b5a2-49def53cb4af" (UID: "f5bf6abd-db25-4f50-b5a2-49def53cb4af"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:16:47.152735 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.152712 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5bf6abd-db25-4f50-b5a2-49def53cb4af-kube-api-access-xcthp" (OuterVolumeSpecName: "kube-api-access-xcthp") pod "f5bf6abd-db25-4f50-b5a2-49def53cb4af" (UID: "f5bf6abd-db25-4f50-b5a2-49def53cb4af"). InnerVolumeSpecName "kube-api-access-xcthp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:16:47.153221 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.153202 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1323e7ab-7964-460c-a2d0-77fbced802b5-kube-api-access-8xltn" (OuterVolumeSpecName: "kube-api-access-8xltn") pod "1323e7ab-7964-460c-a2d0-77fbced802b5" (UID: "1323e7ab-7964-460c-a2d0-77fbced802b5"). InnerVolumeSpecName "kube-api-access-8xltn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:16:47.155649 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.155614 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1323e7ab-7964-460c-a2d0-77fbced802b5-util" (OuterVolumeSpecName: "util") pod "1323e7ab-7964-460c-a2d0-77fbced802b5" (UID: "1323e7ab-7964-460c-a2d0-77fbced802b5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:16:47.156586 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.156566 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5bf6abd-db25-4f50-b5a2-49def53cb4af-util" (OuterVolumeSpecName: "util") pod "f5bf6abd-db25-4f50-b5a2-49def53cb4af" (UID: "f5bf6abd-db25-4f50-b5a2-49def53cb4af"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:16:47.252013 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.251941 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1323e7ab-7964-460c-a2d0-77fbced802b5-util\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:16:47.252013 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.251971 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1323e7ab-7964-460c-a2d0-77fbced802b5-bundle\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:16:47.252013 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.251980 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcthp\" (UniqueName: \"kubernetes.io/projected/f5bf6abd-db25-4f50-b5a2-49def53cb4af-kube-api-access-xcthp\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:16:47.252013 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.251990 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5bf6abd-db25-4f50-b5a2-49def53cb4af-util\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:16:47.252013 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.251998 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5bf6abd-db25-4f50-b5a2-49def53cb4af-bundle\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:16:47.252013 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.252005 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xltn\" (UniqueName: \"kubernetes.io/projected/1323e7ab-7964-460c-a2d0-77fbced802b5-kube-api-access-8xltn\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:16:47.408192 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.408157 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:47.493204 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.493162 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:47.588664 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.588628 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:47.922192 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.922116 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" Jul 16 12:16:47.922192 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.922151 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k" event={"ID":"1323e7ab-7964-460c-a2d0-77fbced802b5","Type":"ContainerDied","Data":"02afd2b595cecd8206f326f6e775c70ae0fa946329009bba7bd92cd288b19e9d"} Jul 16 12:16:47.922689 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.922228 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="02afd2b595cecd8206f326f6e775c70ae0fa946329009bba7bd92cd288b19e9d" Jul 16 12:16:47.924221 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.924149 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" event={"ID":"f5bf6abd-db25-4f50-b5a2-49def53cb4af","Type":"ContainerDied","Data":"ca146f2e315087cfa46244333897c896011582ddcc18fee61381785868890cb7"} Jul 16 12:16:47.924221 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.924193 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca146f2e315087cfa46244333897c896011582ddcc18fee61381785868890cb7" Jul 16 12:16:47.924938 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:47.924571 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s" Jul 16 12:16:48.075778 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.075751 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" Jul 16 12:16:48.077941 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.077922 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" Jul 16 12:16:48.086789 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.086762 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:48.158630 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.158601 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e13484fb-1430-473c-9b88-0d0fbd8e76cc-util\") pod \"e13484fb-1430-473c-9b88-0d0fbd8e76cc\" (UID: \"e13484fb-1430-473c-9b88-0d0fbd8e76cc\") " Jul 16 12:16:48.158783 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.158648 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-bundle\") pod \"c09bf55c-8a63-49eb-9f5a-d09ce780a38a\" (UID: \"c09bf55c-8a63-49eb-9f5a-d09ce780a38a\") " Jul 16 12:16:48.158783 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.158701 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j52dc\" (UniqueName: \"kubernetes.io/projected/e13484fb-1430-473c-9b88-0d0fbd8e76cc-kube-api-access-j52dc\") pod \"e13484fb-1430-473c-9b88-0d0fbd8e76cc\" (UID: \"e13484fb-1430-473c-9b88-0d0fbd8e76cc\") " Jul 16 12:16:48.158783 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.158728 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtn8r\" (UniqueName: \"kubernetes.io/projected/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-kube-api-access-gtn8r\") pod \"c09bf55c-8a63-49eb-9f5a-d09ce780a38a\" (UID: \"c09bf55c-8a63-49eb-9f5a-d09ce780a38a\") " Jul 16 12:16:48.158783 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.158754 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e13484fb-1430-473c-9b88-0d0fbd8e76cc-bundle\") pod \"e13484fb-1430-473c-9b88-0d0fbd8e76cc\" (UID: \"e13484fb-1430-473c-9b88-0d0fbd8e76cc\") " Jul 16 12:16:48.158783 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.158779 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-util\") pod \"c09bf55c-8a63-49eb-9f5a-d09ce780a38a\" (UID: \"c09bf55c-8a63-49eb-9f5a-d09ce780a38a\") " Jul 16 12:16:48.159280 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.159251 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e13484fb-1430-473c-9b88-0d0fbd8e76cc-bundle" (OuterVolumeSpecName: "bundle") pod "e13484fb-1430-473c-9b88-0d0fbd8e76cc" (UID: "e13484fb-1430-473c-9b88-0d0fbd8e76cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:16:48.159280 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.159271 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-bundle" (OuterVolumeSpecName: "bundle") pod "c09bf55c-8a63-49eb-9f5a-d09ce780a38a" (UID: "c09bf55c-8a63-49eb-9f5a-d09ce780a38a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:16:48.160904 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.160875 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e13484fb-1430-473c-9b88-0d0fbd8e76cc-kube-api-access-j52dc" (OuterVolumeSpecName: "kube-api-access-j52dc") pod "e13484fb-1430-473c-9b88-0d0fbd8e76cc" (UID: "e13484fb-1430-473c-9b88-0d0fbd8e76cc"). InnerVolumeSpecName "kube-api-access-j52dc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:16:48.160995 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.160927 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-kube-api-access-gtn8r" (OuterVolumeSpecName: "kube-api-access-gtn8r") pod "c09bf55c-8a63-49eb-9f5a-d09ce780a38a" (UID: "c09bf55c-8a63-49eb-9f5a-d09ce780a38a"). InnerVolumeSpecName "kube-api-access-gtn8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:16:48.164525 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.164501 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-util" (OuterVolumeSpecName: "util") pod "c09bf55c-8a63-49eb-9f5a-d09ce780a38a" (UID: "c09bf55c-8a63-49eb-9f5a-d09ce780a38a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:16:48.165618 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.165597 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e13484fb-1430-473c-9b88-0d0fbd8e76cc-util" (OuterVolumeSpecName: "util") pod "e13484fb-1430-473c-9b88-0d0fbd8e76cc" (UID: "e13484fb-1430-473c-9b88-0d0fbd8e76cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:16:48.188086 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.188027 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:48.259358 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.259336 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j52dc\" (UniqueName: \"kubernetes.io/projected/e13484fb-1430-473c-9b88-0d0fbd8e76cc-kube-api-access-j52dc\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:16:48.259358 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.259358 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtn8r\" (UniqueName: \"kubernetes.io/projected/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-kube-api-access-gtn8r\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:16:48.259358 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.259368 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e13484fb-1430-473c-9b88-0d0fbd8e76cc-bundle\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:16:48.259551 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.259376 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-util\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:16:48.259551 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.259383 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e13484fb-1430-473c-9b88-0d0fbd8e76cc-util\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:16:48.259551 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.259392 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c09bf55c-8a63-49eb-9f5a-d09ce780a38a-bundle\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:16:48.931069 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.931030 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" event={"ID":"e13484fb-1430-473c-9b88-0d0fbd8e76cc","Type":"ContainerDied","Data":"53c127c7b7596ba5ad73ff8670f96b7484179e72c09b2ef1261556c774be79a4"} Jul 16 12:16:48.931069 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.931047 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k" Jul 16 12:16:48.931069 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.931077 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="53c127c7b7596ba5ad73ff8670f96b7484179e72c09b2ef1261556c774be79a4" Jul 16 12:16:48.932891 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.932867 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" event={"ID":"c09bf55c-8a63-49eb-9f5a-d09ce780a38a","Type":"ContainerDied","Data":"88cc9d29ea4629b9c0588144f6f744c9b028db5e5521f11a3295184d08643067"} Jul 16 12:16:48.933001 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.932892 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq" Jul 16 12:16:48.933001 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:48.932905 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88cc9d29ea4629b9c0588144f6f744c9b028db5e5521f11a3295184d08643067" Jul 16 12:16:55.955233 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:55.955188 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:57.860050 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:57.860008 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9df88b6cb-s5p8q"] Jul 16 12:16:58.896212 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:58.896170 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:58.999772 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:58.999728 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:16:59.742233 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:16:59.742198 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:00.729744 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:00.729712 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:05.341687 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:05.341654 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:08.175990 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:08.175946 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:08.510211 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:08.510108 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:08.770645 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:08.770551 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:19.041515 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:19.041483 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:19.158179 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:19.158142 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:20.182256 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:20.182214 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:21.222048 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:21.222016 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:21.346019 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:21.345987 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:21.470687 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:21.470642 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:21.983189 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:21.983152 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:22.064851 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:22.064820 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:22.885431 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:22.885368 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-9df88b6cb-s5p8q" podUID="ab04ad52-6514-473e-9852-6767cbf5291b" containerName="console" containerID="cri-o://79226af62a80c54e242f090a3e82a911020287f1fb2b76ca5d93284b4a62a174" gracePeriod=15 Jul 16 12:17:23.108528 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.108488 2578 generic.go:356] "Generic (PLEG): container finished" podID="ab04ad52-6514-473e-9852-6767cbf5291b" containerID="79226af62a80c54e242f090a3e82a911020287f1fb2b76ca5d93284b4a62a174" exitCode=0 Jul 16 12:17:23.108695 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.108564 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9df88b6cb-s5p8q" event={"ID":"ab04ad52-6514-473e-9852-6767cbf5291b","Type":"ContainerDied","Data":"79226af62a80c54e242f090a3e82a911020287f1fb2b76ca5d93284b4a62a174"} Jul 16 12:17:23.132985 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.132963 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:17:23.254387 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.254304 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-trusted-ca-bundle\") pod \"ab04ad52-6514-473e-9852-6767cbf5291b\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " Jul 16 12:17:23.254387 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.254356 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab04ad52-6514-473e-9852-6767cbf5291b-console-oauth-config\") pod \"ab04ad52-6514-473e-9852-6767cbf5291b\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " Jul 16 12:17:23.254615 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.254454 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk9c4\" (UniqueName: \"kubernetes.io/projected/ab04ad52-6514-473e-9852-6767cbf5291b-kube-api-access-sk9c4\") pod \"ab04ad52-6514-473e-9852-6767cbf5291b\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " Jul 16 12:17:23.254615 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.254482 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-service-ca\") pod \"ab04ad52-6514-473e-9852-6767cbf5291b\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " Jul 16 12:17:23.254615 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.254531 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-oauth-serving-cert\") pod \"ab04ad52-6514-473e-9852-6767cbf5291b\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " Jul 16 12:17:23.254615 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.254560 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-console-config\") pod \"ab04ad52-6514-473e-9852-6767cbf5291b\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " Jul 16 12:17:23.254615 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.254584 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab04ad52-6514-473e-9852-6767cbf5291b-console-serving-cert\") pod \"ab04ad52-6514-473e-9852-6767cbf5291b\" (UID: \"ab04ad52-6514-473e-9852-6767cbf5291b\") " Jul 16 12:17:23.254884 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.254861 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ab04ad52-6514-473e-9852-6767cbf5291b" (UID: "ab04ad52-6514-473e-9852-6767cbf5291b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:17:23.254941 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.254907 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ab04ad52-6514-473e-9852-6767cbf5291b" (UID: "ab04ad52-6514-473e-9852-6767cbf5291b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:17:23.254995 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.254940 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-service-ca" (OuterVolumeSpecName: "service-ca") pod "ab04ad52-6514-473e-9852-6767cbf5291b" (UID: "ab04ad52-6514-473e-9852-6767cbf5291b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:17:23.255041 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.255003 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-console-config" (OuterVolumeSpecName: "console-config") pod "ab04ad52-6514-473e-9852-6767cbf5291b" (UID: "ab04ad52-6514-473e-9852-6767cbf5291b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:17:23.256633 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.256612 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab04ad52-6514-473e-9852-6767cbf5291b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ab04ad52-6514-473e-9852-6767cbf5291b" (UID: "ab04ad52-6514-473e-9852-6767cbf5291b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:17:23.256734 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.256646 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab04ad52-6514-473e-9852-6767cbf5291b-kube-api-access-sk9c4" (OuterVolumeSpecName: "kube-api-access-sk9c4") pod "ab04ad52-6514-473e-9852-6767cbf5291b" (UID: "ab04ad52-6514-473e-9852-6767cbf5291b"). InnerVolumeSpecName "kube-api-access-sk9c4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:17:23.256734 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.256679 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab04ad52-6514-473e-9852-6767cbf5291b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ab04ad52-6514-473e-9852-6767cbf5291b" (UID: "ab04ad52-6514-473e-9852-6767cbf5291b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:17:23.355730 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.355695 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk9c4\" (UniqueName: \"kubernetes.io/projected/ab04ad52-6514-473e-9852-6767cbf5291b-kube-api-access-sk9c4\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:17:23.355927 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.355905 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-service-ca\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:17:23.356018 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.355933 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-oauth-serving-cert\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:17:23.356018 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.355947 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-console-config\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:17:23.356018 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.355973 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab04ad52-6514-473e-9852-6767cbf5291b-console-serving-cert\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:17:23.356018 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.355987 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab04ad52-6514-473e-9852-6767cbf5291b-trusted-ca-bundle\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:17:23.356018 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.355999 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab04ad52-6514-473e-9852-6767cbf5291b-console-oauth-config\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:17:23.434012 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.433973 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:23.512464 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.512381 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:23.612804 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:23.612770 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:24.122535 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:24.122494 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9df88b6cb-s5p8q" event={"ID":"ab04ad52-6514-473e-9852-6767cbf5291b","Type":"ContainerDied","Data":"ccc2c6093960bd3a96e4beef0aad64d92f3101e98e0a21a675e4bcc65e40ee8a"} Jul 16 12:17:24.122905 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:24.122538 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9df88b6cb-s5p8q" Jul 16 12:17:24.123377 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:24.122575 2578 scope.go:117] "RemoveContainer" containerID="79226af62a80c54e242f090a3e82a911020287f1fb2b76ca5d93284b4a62a174" Jul 16 12:17:24.148335 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:24.148311 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9df88b6cb-s5p8q"] Jul 16 12:17:24.153688 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:24.153665 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9df88b6cb-s5p8q"] Jul 16 12:17:25.383487 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:25.383447 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:25.479349 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:25.479316 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab04ad52-6514-473e-9852-6767cbf5291b" path="/var/lib/kubelet/pods/ab04ad52-6514-473e-9852-6767cbf5291b/volumes" Jul 16 12:17:25.484904 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:25.484879 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:30.677296 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:30.677271 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:17:30.679248 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:30.679226 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:17:30.684155 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:30.684135 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:17:30.685688 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:30.685668 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:17:30.814193 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:30.814151 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:30.914994 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:30.914961 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:37.169140 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:37.169103 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:37.273728 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:37.273695 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:40.651476 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:40.651435 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:40.754248 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:40.754219 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:40.832131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:40.832077 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:43.033050 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:43.033009 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:43.132381 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:43.132339 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:48.355125 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:48.355088 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:48.454468 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:48.454425 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:55.057222 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:55.057142 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:55.510755 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:55.510659 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:58.827020 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:58.826986 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:58.892936 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:58.892904 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:59.036459 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:59.036423 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:17:59.109680 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:17:59.109592 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:02.103383 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:02.103348 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:02.187244 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:02.187203 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:02.625094 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:02.625057 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:02.685760 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:02.685729 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:06.045946 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:06.045907 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:06.268975 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:06.268940 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:07.704646 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:07.704608 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:07.819114 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:07.819082 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:21.047051 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:21.047012 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:22.823579 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:22.823528 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:23.815144 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:23.815113 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:23.910861 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:23.910827 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:24.010229 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.010194 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:24.304459 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304426 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:18:24.304770 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304757 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab04ad52-6514-473e-9852-6767cbf5291b" containerName="console" Jul 16 12:18:24.304814 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304772 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab04ad52-6514-473e-9852-6767cbf5291b" containerName="console" Jul 16 12:18:24.304814 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304780 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1323e7ab-7964-460c-a2d0-77fbced802b5" containerName="util" Jul 16 12:18:24.304814 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304787 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1323e7ab-7964-460c-a2d0-77fbced802b5" containerName="util" Jul 16 12:18:24.304814 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304798 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e13484fb-1430-473c-9b88-0d0fbd8e76cc" containerName="util" Jul 16 12:18:24.304814 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304806 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e13484fb-1430-473c-9b88-0d0fbd8e76cc" containerName="util" Jul 16 12:18:24.304968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304817 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c09bf55c-8a63-49eb-9f5a-d09ce780a38a" containerName="extract" Jul 16 12:18:24.304968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304825 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c09bf55c-8a63-49eb-9f5a-d09ce780a38a" containerName="extract" Jul 16 12:18:24.304968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304843 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5bf6abd-db25-4f50-b5a2-49def53cb4af" containerName="pull" Jul 16 12:18:24.304968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304851 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5bf6abd-db25-4f50-b5a2-49def53cb4af" containerName="pull" Jul 16 12:18:24.304968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304863 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5bf6abd-db25-4f50-b5a2-49def53cb4af" containerName="util" Jul 16 12:18:24.304968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304871 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5bf6abd-db25-4f50-b5a2-49def53cb4af" containerName="util" Jul 16 12:18:24.304968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304881 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1323e7ab-7964-460c-a2d0-77fbced802b5" containerName="extract" Jul 16 12:18:24.304968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304890 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1323e7ab-7964-460c-a2d0-77fbced802b5" containerName="extract" Jul 16 12:18:24.304968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304902 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c09bf55c-8a63-49eb-9f5a-d09ce780a38a" containerName="pull" Jul 16 12:18:24.304968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304910 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c09bf55c-8a63-49eb-9f5a-d09ce780a38a" containerName="pull" Jul 16 12:18:24.304968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304933 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1323e7ab-7964-460c-a2d0-77fbced802b5" containerName="pull" Jul 16 12:18:24.304968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304941 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1323e7ab-7964-460c-a2d0-77fbced802b5" containerName="pull" Jul 16 12:18:24.304968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304957 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c09bf55c-8a63-49eb-9f5a-d09ce780a38a" containerName="util" Jul 16 12:18:24.304968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304964 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c09bf55c-8a63-49eb-9f5a-d09ce780a38a" containerName="util" Jul 16 12:18:24.305427 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304977 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5bf6abd-db25-4f50-b5a2-49def53cb4af" containerName="extract" Jul 16 12:18:24.305427 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304984 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5bf6abd-db25-4f50-b5a2-49def53cb4af" containerName="extract" Jul 16 12:18:24.305427 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.304996 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e13484fb-1430-473c-9b88-0d0fbd8e76cc" containerName="pull" Jul 16 12:18:24.305427 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.305004 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e13484fb-1430-473c-9b88-0d0fbd8e76cc" containerName="pull" Jul 16 12:18:24.305427 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.305015 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e13484fb-1430-473c-9b88-0d0fbd8e76cc" containerName="extract" Jul 16 12:18:24.305427 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.305023 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e13484fb-1430-473c-9b88-0d0fbd8e76cc" containerName="extract" Jul 16 12:18:24.305427 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.305106 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab04ad52-6514-473e-9852-6767cbf5291b" containerName="console" Jul 16 12:18:24.305427 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.305115 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5bf6abd-db25-4f50-b5a2-49def53cb4af" containerName="extract" Jul 16 12:18:24.305427 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.305121 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e13484fb-1430-473c-9b88-0d0fbd8e76cc" containerName="extract" Jul 16 12:18:24.305427 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.305128 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c09bf55c-8a63-49eb-9f5a-d09ce780a38a" containerName="extract" Jul 16 12:18:24.305427 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.305133 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1323e7ab-7964-460c-a2d0-77fbced802b5" containerName="extract" Jul 16 12:18:24.308242 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.308226 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-cs4r8" Jul 16 12:18:24.311015 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.310993 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 16 12:18:24.311313 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.311298 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 12:18:24.311816 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.311803 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 12:18:24.311871 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.311831 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tf89p\"" Jul 16 12:18:24.319013 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.318991 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:18:24.348326 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.348287 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:18:24.398949 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.398921 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f1e16a12-c1d6-49c2-bc1c-325dccb707bb-config-file\") pod \"limitador-limitador-54d6d988d5-cs4r8\" (UID: \"f1e16a12-c1d6-49c2-bc1c-325dccb707bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cs4r8" Jul 16 12:18:24.399104 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.398966 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h78hg\" (UniqueName: \"kubernetes.io/projected/f1e16a12-c1d6-49c2-bc1c-325dccb707bb-kube-api-access-h78hg\") pod \"limitador-limitador-54d6d988d5-cs4r8\" (UID: \"f1e16a12-c1d6-49c2-bc1c-325dccb707bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cs4r8" Jul 16 12:18:24.499696 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.499665 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f1e16a12-c1d6-49c2-bc1c-325dccb707bb-config-file\") pod \"limitador-limitador-54d6d988d5-cs4r8\" (UID: \"f1e16a12-c1d6-49c2-bc1c-325dccb707bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cs4r8" Jul 16 12:18:24.499856 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.499713 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h78hg\" (UniqueName: \"kubernetes.io/projected/f1e16a12-c1d6-49c2-bc1c-325dccb707bb-kube-api-access-h78hg\") pod \"limitador-limitador-54d6d988d5-cs4r8\" (UID: \"f1e16a12-c1d6-49c2-bc1c-325dccb707bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cs4r8" Jul 16 12:18:24.500274 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.500256 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f1e16a12-c1d6-49c2-bc1c-325dccb707bb-config-file\") pod \"limitador-limitador-54d6d988d5-cs4r8\" (UID: \"f1e16a12-c1d6-49c2-bc1c-325dccb707bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cs4r8" Jul 16 12:18:24.508230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.508208 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h78hg\" (UniqueName: \"kubernetes.io/projected/f1e16a12-c1d6-49c2-bc1c-325dccb707bb-kube-api-access-h78hg\") pod \"limitador-limitador-54d6d988d5-cs4r8\" (UID: \"f1e16a12-c1d6-49c2-bc1c-325dccb707bb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cs4r8" Jul 16 12:18:24.619489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.619383 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-cs4r8" Jul 16 12:18:24.681028 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.680995 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:24.774425 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.774389 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:18:24.776740 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:18:24.776707 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e16a12_c1d6_49c2_bc1c_325dccb707bb.slice/crio-9a715263dc42a4f7b766b39b6f854ba59cada06fc5064972561e656c92610e70 WatchSource:0}: Error finding container 9a715263dc42a4f7b766b39b6f854ba59cada06fc5064972561e656c92610e70: Status 404 returned error can't find the container with id 9a715263dc42a4f7b766b39b6f854ba59cada06fc5064972561e656c92610e70 Jul 16 12:18:24.987612 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:24.987512 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:25.431339 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:25.431305 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-cs4r8" event={"ID":"f1e16a12-c1d6-49c2-bc1c-325dccb707bb","Type":"ContainerStarted","Data":"9a715263dc42a4f7b766b39b6f854ba59cada06fc5064972561e656c92610e70"} Jul 16 12:18:28.452811 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:28.452710 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-cs4r8" event={"ID":"f1e16a12-c1d6-49c2-bc1c-325dccb707bb","Type":"ContainerStarted","Data":"ac72289591dec2ce822dc81c4647b070e74a4bdba59e1657eabacef0b5a75888"} Jul 16 12:18:28.453158 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:28.452809 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-cs4r8" Jul 16 12:18:28.485232 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:28.485187 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-cs4r8" podStartSLOduration=1.205277244 podStartE2EDuration="4.485172255s" totalImagesPullingTime="3.279895011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:18:28.485149743 +0000 UTC m=+658.248519206" watchObservedRunningTime="2026-07-16 12:18:28.485172255 +0000 UTC m=+658.248541744" Jul 16 12:18:39.459132 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:39.459101 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-cs4r8" Jul 16 12:18:45.227722 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:45.227675 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:45.339062 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:45.339028 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:46.143444 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:46.143386 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:57.996279 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:57.996244 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:18:58.441789 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:18:58.441758 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:19:06.527488 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:19:06.527454 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:19:06.622764 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:19:06.622722 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:19:06.690924 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:19:06.690892 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:19:07.166061 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:19:07.165993 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:19:10.702073 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:19:10.702037 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:19:10.802862 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:19:10.802833 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:19:34.131839 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:19:34.131753 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:19:39.553080 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:19:39.553044 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:19:39.646653 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:19:39.646614 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:19:39.948829 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:19:39.948749 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:19:40.367552 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:19:40.367507 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:19:40.510598 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:19:40.510570 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:19:40.608158 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:19:40.608122 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:19:40.724858 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:19:40.724780 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:19:52.005223 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:19:52.005189 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:19:52.100431 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:19:52.100391 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:20:10.336042 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:20:10.336009 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:20:11.526039 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:20:11.526004 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:20:12.030177 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:20:12.030143 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:20:23.638806 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:20:23.638768 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:20:24.550601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:20:24.550566 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:20:30.531942 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:20:30.531909 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:20:31.724650 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:20:31.724615 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:20:37.788669 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:20:37.788636 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:20:39.331046 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:20:39.331011 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:20:53.108712 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:20:53.108599 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:20:54.305616 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:20:54.305581 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:20:54.820744 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:20:54.820713 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:20:56.554923 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:20:56.554885 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:21:05.026299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:21:05.026266 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:21:12.880126 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:21:12.880088 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:22:03.643058 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:22:03.643014 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:22:03.739260 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:22:03.739226 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:22:04.935090 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:22:04.935056 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:22:05.041608 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:22:05.041576 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:22:30.717400 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:22:30.717330 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:22:30.717400 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:22:30.717374 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:22:30.724353 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:22:30.724326 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:22:30.724537 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:22:30.724432 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:22:33.934973 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:22:33.934928 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:22:43.624679 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:22:43.624645 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:22:54.536975 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:22:54.536937 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:23:03.279741 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:23:03.279701 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:23:11.805024 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:23:11.804989 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:24:09.677536 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:24:09.677440 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:24:11.203134 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:24:11.203102 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:24:37.623072 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:24:37.623039 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:24:47.548542 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:24:47.548508 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:24:47.947179 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:24:47.947088 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:24:48.048133 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:24:48.048094 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:24:48.191282 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:24:48.191253 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:24:54.380618 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:24:54.380582 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:24:57.428336 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:24:57.428296 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:24:57.490187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:24:57.490158 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:25:01.048648 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:25:01.048617 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:25:01.130688 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:25:01.130658 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:25:20.053702 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:25:20.053662 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:25:32.817201 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:25:32.817116 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:25:40.337655 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:25:40.337622 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:25:48.332745 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:25:48.332705 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:26:00.550823 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:26:00.550783 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:26:06.191861 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:26:06.191828 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:26:10.425948 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:26:10.425912 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:26:21.332942 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:26:21.332910 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:26:31.335639 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:26:31.335435 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:26:53.434244 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:26:53.434160 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:27:05.438808 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:27:05.438772 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:27:30.023734 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:27:30.023699 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:27:30.752003 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:27:30.751976 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:27:30.755730 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:27:30.755705 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:27:30.757677 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:27:30.757657 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:27:30.761685 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:27:30.761667 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:27:41.532676 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:27:41.532644 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:27:57.727330 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:27:57.727293 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:27:59.160821 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:27:59.160787 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:28:09.925750 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:28:09.925710 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:28:26.425693 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:28:26.425603 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:28:37.851731 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:28:37.851690 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:29:33.885214 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:29:33.885178 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:30:25.752430 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:30:25.752381 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:30:36.654319 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:30:36.654287 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:30:47.948309 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:30:47.948264 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:30:58.645722 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:30:58.645686 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:31:10.961668 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:31:10.961625 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:31:21.303595 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:31:21.303558 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:31:30.194008 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:31:30.193923 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:31:41.928753 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:31:41.928716 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:31:52.253869 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:31:52.253826 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:32:00.947932 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:32:00.947892 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:32:16.942211 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:32:16.942171 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:32:30.787463 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:32:30.787434 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:32:30.791952 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:32:30.791927 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:32:30.793606 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:32:30.793587 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:32:30.797793 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:32:30.797772 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:32:32.327224 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:32:32.327193 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:32:42.036897 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:32:42.036864 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:32:52.321152 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:32:52.321066 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:33:02.142869 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:33:02.142836 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:33:07.941066 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:33:07.941029 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:33:27.666321 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:33:27.666287 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:33:37.837966 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:33:37.837932 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:33:47.144301 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:33:47.144268 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:33:55.224575 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:33:55.224545 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:34:06.457716 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:34:06.457682 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:34:16.436845 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:34:16.436811 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:34:17.360057 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:34:17.360021 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:34:27.427575 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:34:27.427500 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:34:39.331686 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:34:39.331645 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:34:49.056034 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:34:49.056002 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:35:00.364357 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:35:00.364317 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:35:13.253277 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:35:13.253240 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:35:22.629578 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:35:22.629547 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:35:35.838702 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:35:35.838664 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:35:48.443855 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:35:48.443820 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:35:57.863681 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:35:57.863593 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:36:06.929937 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:06.929902 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:36:12.558583 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:12.558550 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:12.632938 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:12.632899 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:12.731725 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:12.731690 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:13.040811 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:13.040777 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:20.512292 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.512215 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd"] Jul 16 12:36:20.515789 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.515768 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd" Jul 16 12:36:20.518166 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.518146 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c7727fcf-serving-cert\"" Jul 16 12:36:20.518494 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.518478 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 16 12:36:20.519514 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.519495 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-x8mkx\"" Jul 16 12:36:20.519640 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.519565 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 12:36:20.534404 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.534382 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd"] Jul 16 12:36:20.578214 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.578184 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxq7b\" (UniqueName: \"kubernetes.io/projected/be2bf342-ab6e-498f-b302-aeeb783fe9d4-kube-api-access-sxq7b\") pod \"maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd\" (UID: \"be2bf342-ab6e-498f-b302-aeeb783fe9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd" Jul 16 12:36:20.578339 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.578231 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be2bf342-ab6e-498f-b302-aeeb783fe9d4-openshift-service-ca\") pod \"maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd\" (UID: \"be2bf342-ab6e-498f-b302-aeeb783fe9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd" Jul 16 12:36:20.578339 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.578303 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be2bf342-ab6e-498f-b302-aeeb783fe9d4-maas-api-tls\") pod \"maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd\" (UID: \"be2bf342-ab6e-498f-b302-aeeb783fe9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd" Jul 16 12:36:20.679000 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.678965 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be2bf342-ab6e-498f-b302-aeeb783fe9d4-maas-api-tls\") pod \"maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd\" (UID: \"be2bf342-ab6e-498f-b302-aeeb783fe9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd" Jul 16 12:36:20.679178 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.679045 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxq7b\" (UniqueName: \"kubernetes.io/projected/be2bf342-ab6e-498f-b302-aeeb783fe9d4-kube-api-access-sxq7b\") pod \"maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd\" (UID: \"be2bf342-ab6e-498f-b302-aeeb783fe9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd" Jul 16 12:36:20.679178 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.679086 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be2bf342-ab6e-498f-b302-aeeb783fe9d4-openshift-service-ca\") pod \"maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd\" (UID: \"be2bf342-ab6e-498f-b302-aeeb783fe9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd" Jul 16 12:36:20.679782 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.679762 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be2bf342-ab6e-498f-b302-aeeb783fe9d4-openshift-service-ca\") pod \"maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd\" (UID: \"be2bf342-ab6e-498f-b302-aeeb783fe9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd" Jul 16 12:36:20.681405 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.681383 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be2bf342-ab6e-498f-b302-aeeb783fe9d4-maas-api-tls\") pod \"maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd\" (UID: \"be2bf342-ab6e-498f-b302-aeeb783fe9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd" Jul 16 12:36:20.687856 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.687834 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxq7b\" (UniqueName: \"kubernetes.io/projected/be2bf342-ab6e-498f-b302-aeeb783fe9d4-kube-api-access-sxq7b\") pod \"maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd\" (UID: \"be2bf342-ab6e-498f-b302-aeeb783fe9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd" Jul 16 12:36:20.825854 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.825820 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd" Jul 16 12:36:20.962977 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.962942 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd"] Jul 16 12:36:20.966009 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:36:20.965985 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe2bf342_ab6e_498f_b302_aeeb783fe9d4.slice/crio-07306ef712175317085ca2be28fd1e60593cf4442548dc062c286aa0b8cc5c78 WatchSource:0}: Error finding container 07306ef712175317085ca2be28fd1e60593cf4442548dc062c286aa0b8cc5c78: Status 404 returned error can't find the container with id 07306ef712175317085ca2be28fd1e60593cf4442548dc062c286aa0b8cc5c78 Jul 16 12:36:20.967328 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:20.967309 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 12:36:21.463915 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:21.463870 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:21.562683 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:21.562651 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:21.966771 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:21.966670 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd" event={"ID":"be2bf342-ab6e-498f-b302-aeeb783fe9d4","Type":"ContainerStarted","Data":"07306ef712175317085ca2be28fd1e60593cf4442548dc062c286aa0b8cc5c78"} Jul 16 12:36:23.587914 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:23.587882 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:23.825468 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:23.825367 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:36:23.979634 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:23.979600 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd" event={"ID":"be2bf342-ab6e-498f-b302-aeeb783fe9d4","Type":"ContainerStarted","Data":"211414ae0f3d94ff5a0680bcad2539174669f10f041b1b9fe600d47884370c0b"} Jul 16 12:36:23.979815 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:23.979702 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd" Jul 16 12:36:24.001520 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:24.001475 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd" podStartSLOduration=1.669201128 podStartE2EDuration="4.001459938s" totalImagesPullingTime="2.33225881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:36:23.999819483 +0000 UTC m=+1733.763188999" watchObservedRunningTime="2026-07-16 12:36:24.001459938 +0000 UTC m=+1733.764829401" Jul 16 12:36:26.647865 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:26.647825 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:26.765482 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:26.765444 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:28.360571 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:28.360527 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:29.991213 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:29.991186 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd" Jul 16 12:36:32.924895 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:32.924857 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:36:33.070264 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:33.070228 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:33.171039 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:33.171002 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:33.335667 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:33.335633 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:33.445255 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:33.445221 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:33.612244 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:33.612154 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:33.679777 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:33.679741 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:35.484811 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:35.484781 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:35.548829 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:35.548798 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:36.893534 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:36.893499 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:39.601628 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:39.601591 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:39.706349 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:39.706318 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:41.159475 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:41.159407 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:46.412914 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:46.412873 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:46.534827 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:46.534791 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:46.647745 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:46.647711 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:46.770580 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:46.770494 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:46.892008 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:46.891968 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:47.085426 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:47.085376 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:47.399797 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:47.399709 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:48.131363 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:48.131325 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:48.278789 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:48.278741 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:49.897681 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:49.897646 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:50.824531 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:50.824496 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:36:52.463015 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:52.462979 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:52.563723 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:52.563684 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:54.200159 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:54.200123 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:59.119934 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:59.119900 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:59.270400 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:59.270366 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:36:59.852525 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:36:59.852489 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:37:00.089049 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:00.089015 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:37:00.201018 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:00.200925 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:37:00.302283 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:00.302254 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:37:01.883923 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:01.883884 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:37:08.558523 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:08.558487 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:37:08.632541 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:08.632503 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:37:08.704222 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:08.704175 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:37:08.804563 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:08.804527 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:37:08.925196 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:08.925094 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:37:09.462786 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:09.462744 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:37:30.824847 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:30.824820 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:37:30.831235 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:30.831213 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:37:30.831626 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:30.831604 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:37:30.837020 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:30.837002 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:37:35.584763 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:35.584729 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:37:41.777218 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:41.777177 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:37:41.831739 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:41.831704 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:37:41.935065 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:41.935024 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:37:42.031253 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:42.031210 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:37:58.081298 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:37:58.081263 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:04.635338 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:04.635299 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:04.750245 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:04.750208 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:04.843938 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:04.843904 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:14.197270 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:14.197236 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:14.270563 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:14.270532 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:15.429374 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:15.429330 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:16.364669 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:16.364628 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:22.875009 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:22.874977 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:23.044249 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:23.044216 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:23.133127 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:23.133044 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:23.236956 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:23.236924 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:24.155486 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:24.155446 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:24.264148 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:24.264113 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:24.468613 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:24.468514 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:25.465701 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:25.465664 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:38:26.426105 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:26.426067 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:33.043185 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:33.043142 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:33.137523 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:33.137484 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:33.815243 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:33.815208 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:33.943389 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:33.943356 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:38:40.512518 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:40.512480 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:40.602612 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:40.602586 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:40.798344 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:40.798303 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:41.143949 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:41.143867 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:41.233919 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:41.233884 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:43.027360 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:43.027321 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:38:43.414445 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:43.414395 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:43.512435 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:43.512380 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:43.603890 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:43.603849 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:43.702296 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:43.702194 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:44.108242 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:44.108204 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:44.208098 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:44.208047 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:47.138904 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:47.138874 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:38:48.991977 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:48.991944 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:49.747201 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:49.747169 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:49.889681 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:49.889634 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:50.253448 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:50.253394 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:56.810561 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:56.810469 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:58.406091 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:58.406032 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:38:58.507395 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:38:58.507349 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:03.902966 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:03.902929 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:04.003892 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:04.003861 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:04.193996 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:04.193909 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:04.469112 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:04.469023 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:04.609934 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:04.609903 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:04.708837 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:04.708800 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:06.176321 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:06.176284 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:07.240776 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:07.240739 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:39:11.078265 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:11.078228 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:11.137959 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:11.137921 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:11.262901 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:11.262865 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:11.355771 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:11.355690 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:11.542247 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:11.542208 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:18.023438 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:18.023377 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:18.123853 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:18.123821 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:26.227724 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:26.227683 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:31.528787 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:31.528746 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:39:35.053737 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:35.053702 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:35.153524 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:35.153479 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:35.254748 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:35.254715 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:35.486037 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:35.485944 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:35.587174 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:35.587145 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:36.631404 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:36.631372 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:42.419720 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:42.419676 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:42.487976 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:42.487936 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:42.632137 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:42.632094 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:42.780236 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:42.780196 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:43.249358 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:43.249265 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:39:53.333763 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:53.333727 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:39:53.727799 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:39:53.727708 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:14.839521 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:14.839470 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:21.070807 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.070775 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:21.178925 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.178879 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:21.462680 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.462578 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:21.802688 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.802659 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs"] Jul 16 12:40:21.806113 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.806094 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:21.808631 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.808612 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-28d017-openshift-default-dockercfg-pmdj5\"" Jul 16 12:40:21.822574 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.822545 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs"] Jul 16 12:40:21.830104 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.830076 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:21.990605 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.990562 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q8gg\" (UniqueName: \"kubernetes.io/projected/36148965-e677-471b-986e-f5b07d5db457-kube-api-access-8q8gg\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:21.990605 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.990609 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36148965-e677-471b-986e-f5b07d5db457-istiod-ca-cert\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:21.990819 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.990665 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-workload-certs\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:21.990819 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.990685 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-istio-data\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:21.990819 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.990721 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-workload-socket\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:21.990819 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.990739 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-credential-socket\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:21.990819 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.990758 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-istio-envoy\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:21.990819 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.990778 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36148965-e677-471b-986e-f5b07d5db457-istio-token\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:21.991057 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:21.990853 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36148965-e677-471b-986e-f5b07d5db457-istio-podinfo\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.091748 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.091662 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-workload-certs\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.091748 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.091700 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-istio-data\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.091748 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.091721 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-workload-socket\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.091748 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.091737 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-credential-socket\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.092335 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.091757 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-istio-envoy\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.092335 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.091772 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36148965-e677-471b-986e-f5b07d5db457-istio-token\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.092335 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.091821 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36148965-e677-471b-986e-f5b07d5db457-istio-podinfo\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.092335 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.091865 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q8gg\" (UniqueName: \"kubernetes.io/projected/36148965-e677-471b-986e-f5b07d5db457-kube-api-access-8q8gg\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.092335 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.091893 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36148965-e677-471b-986e-f5b07d5db457-istiod-ca-cert\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.092335 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.092155 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-istio-data\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.092335 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.092178 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-workload-socket\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.092335 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.092200 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-workload-certs\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.092652 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.092445 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-credential-socket\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.092652 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.092527 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36148965-e677-471b-986e-f5b07d5db457-istiod-ca-cert\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.094647 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.094626 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-istio-envoy\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.094911 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.094890 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36148965-e677-471b-986e-f5b07d5db457-istio-podinfo\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.100842 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.100823 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36148965-e677-471b-986e-f5b07d5db457-istio-token\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.101116 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.101098 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q8gg\" (UniqueName: \"kubernetes.io/projected/36148965-e677-471b-986e-f5b07d5db457-kube-api-access-8q8gg\") pod \"e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.116975 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.116957 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:22.262265 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.262061 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs"] Jul 16 12:40:22.265170 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:40:22.265138 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36148965_e677_471b_986e_f5b07d5db457.slice/crio-c5df35d20d793855536d35098ddc368573bc87eff17f6508ee77fd235de2901b WatchSource:0}: Error finding container c5df35d20d793855536d35098ddc368573bc87eff17f6508ee77fd235de2901b: Status 404 returned error can't find the container with id c5df35d20d793855536d35098ddc368573bc87eff17f6508ee77fd235de2901b Jul 16 12:40:22.383974 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:22.383856 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:23.227818 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:23.227772 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" event={"ID":"36148965-e677-471b-986e-f5b07d5db457","Type":"ContainerStarted","Data":"c5df35d20d793855536d35098ddc368573bc87eff17f6508ee77fd235de2901b"} Jul 16 12:40:24.958117 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:24.958080 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:40:24.958457 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:24.958155 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:40:24.958457 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:24.958185 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:40:25.241302 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:25.241213 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" event={"ID":"36148965-e677-471b-986e-f5b07d5db457","Type":"ContainerStarted","Data":"b9bfe87bb66ea852dd29d2beef34dcc43b8e0c9a29b33dfa01601a877d34ab6a"} Jul 16 12:40:25.266755 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:25.266692 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" podStartSLOduration=1.5761862469999999 podStartE2EDuration="4.266671798s" totalImagesPullingTime="2.690485551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:40:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:40:25.263923205 +0000 UTC m=+1975.027292669" watchObservedRunningTime="2026-07-16 12:40:25.266671798 +0000 UTC m=+1975.030041262" Jul 16 12:40:26.118164 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:26.118122 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:26.122997 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:26.122974 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:26.247570 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:26.247532 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:26.248718 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:26.248696 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:26.300361 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:26.300323 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:26.401532 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:26.401449 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:26.496321 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:26.496287 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:26.903006 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:26.902971 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:27.067398 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:27.067360 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:27.183269 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:27.183173 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:31.134397 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:31.134360 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:40:31.928754 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:31.928721 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:33.239309 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:33.239267 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:33.329141 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:33.329094 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:35.032122 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:35.032086 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:35.141462 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:35.141028 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:35.232159 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:35.232124 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:35.324969 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:35.324875 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:35.747465 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:35.747363 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:36.679646 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:36.679603 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:40.331945 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:40.331913 2578 generic.go:356] "Generic (PLEG): container finished" podID="36148965-e677-471b-986e-f5b07d5db457" containerID="b9bfe87bb66ea852dd29d2beef34dcc43b8e0c9a29b33dfa01601a877d34ab6a" exitCode=0 Jul 16 12:40:40.332460 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:40.331989 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" event={"ID":"36148965-e677-471b-986e-f5b07d5db457","Type":"ContainerDied","Data":"b9bfe87bb66ea852dd29d2beef34dcc43b8e0c9a29b33dfa01601a877d34ab6a"} Jul 16 12:40:40.332460 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:40.332378 2578 scope.go:117] "RemoveContainer" containerID="b9bfe87bb66ea852dd29d2beef34dcc43b8e0c9a29b33dfa01601a877d34ab6a" Jul 16 12:40:40.333377 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:40.333346 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:40:40.333475 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:40.333424 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:40:40.333475 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:40.333455 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:40:40.550330 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:40.550288 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:41.247818 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:41.247770 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:41.343299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:41.343263 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" event={"ID":"36148965-e677-471b-986e-f5b07d5db457","Type":"ContainerStarted","Data":"c22066ce45d7be415ac16f2e923d9e4da0adeaf510a046464a90fbf35bb45c38"} Jul 16 12:40:42.119454 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:42.119397 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:42.119687 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:42.119594 2578 patch_prober.go:28] interesting pod/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 16 12:40:42.119687 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:42.119641 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" podUID="36148965-e677-471b-986e-f5b07d5db457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 16 12:40:43.118077 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:43.118031 2578 patch_prober.go:28] interesting pod/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 16 12:40:43.118625 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:43.118108 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" podUID="36148965-e677-471b-986e-f5b07d5db457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 16 12:40:44.117758 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:44.117712 2578 patch_prober.go:28] interesting pod/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 16 12:40:44.117966 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:44.117781 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" podUID="36148965-e677-471b-986e-f5b07d5db457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 16 12:40:44.742312 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:44.742272 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:40:45.117798 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:45.117759 2578 patch_prober.go:28] interesting pod/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 16 12:40:45.118142 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:45.118110 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" podUID="36148965-e677-471b-986e-f5b07d5db457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 16 12:40:46.121855 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:46.121824 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:46.122264 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:46.122206 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:46.122852 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:46.122832 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:40:48.115876 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:48.115836 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:48.218363 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:48.218319 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:56.150294 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:56.150259 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:57.750376 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:57.750342 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:57.850702 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:57.850655 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:40:59.631695 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:40:59.631661 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:41:07.378193 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:41:07.378135 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:41:23.534567 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:41:23.534534 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:41:27.939497 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:41:27.939448 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:41:40.831822 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:41:40.831787 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:41:48.292690 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:41:48.292657 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:41:49.645009 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:41:49.644967 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:41:49.883897 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:41:49.883866 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:41:54.964940 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:41:54.964850 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:41:56.730926 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:41:56.730893 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:42:17.798191 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:17.798149 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:20.583789 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:20.583748 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:30.865140 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:30.865114 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:42:30.871058 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:30.871034 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:42:30.871221 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:30.871101 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:42:30.877616 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:30.877593 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:42:31.955253 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:31.955216 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:32.310938 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:32.310897 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:32.311994 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:32.311971 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs"] Jul 16 12:42:32.312214 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:32.312182 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" podUID="36148965-e677-471b-986e-f5b07d5db457" containerName="istio-proxy" containerID="cri-o://c22066ce45d7be415ac16f2e923d9e4da0adeaf510a046464a90fbf35bb45c38" gracePeriod=30 Jul 16 12:42:32.542715 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:32.542679 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:33.110883 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:33.110847 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:35.921891 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:35.921846 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:42:37.570614 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.570587 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:42:37.713714 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.713619 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36148965-e677-471b-986e-f5b07d5db457-istiod-ca-cert\") pod \"36148965-e677-471b-986e-f5b07d5db457\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " Jul 16 12:42:37.713714 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.713657 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-istio-envoy\") pod \"36148965-e677-471b-986e-f5b07d5db457\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " Jul 16 12:42:37.713714 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.713682 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36148965-e677-471b-986e-f5b07d5db457-istio-token\") pod \"36148965-e677-471b-986e-f5b07d5db457\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " Jul 16 12:42:37.713714 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.713712 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-istio-data\") pod \"36148965-e677-471b-986e-f5b07d5db457\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " Jul 16 12:42:37.714061 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.713744 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-credential-socket\") pod \"36148965-e677-471b-986e-f5b07d5db457\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " Jul 16 12:42:37.714061 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.713788 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-workload-socket\") pod \"36148965-e677-471b-986e-f5b07d5db457\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " Jul 16 12:42:37.714061 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.713812 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8q8gg\" (UniqueName: \"kubernetes.io/projected/36148965-e677-471b-986e-f5b07d5db457-kube-api-access-8q8gg\") pod \"36148965-e677-471b-986e-f5b07d5db457\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " Jul 16 12:42:37.714061 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.713870 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-workload-certs\") pod \"36148965-e677-471b-986e-f5b07d5db457\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " Jul 16 12:42:37.714061 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.713897 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36148965-e677-471b-986e-f5b07d5db457-istio-podinfo\") pod \"36148965-e677-471b-986e-f5b07d5db457\" (UID: \"36148965-e677-471b-986e-f5b07d5db457\") " Jul 16 12:42:37.714323 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.714085 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "36148965-e677-471b-986e-f5b07d5db457" (UID: "36148965-e677-471b-986e-f5b07d5db457"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:42:37.714323 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.714104 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "36148965-e677-471b-986e-f5b07d5db457" (UID: "36148965-e677-471b-986e-f5b07d5db457"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:42:37.714323 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.714129 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36148965-e677-471b-986e-f5b07d5db457-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "36148965-e677-471b-986e-f5b07d5db457" (UID: "36148965-e677-471b-986e-f5b07d5db457"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:42:37.714323 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.714195 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-workload-socket\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:42:37.714323 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.714213 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36148965-e677-471b-986e-f5b07d5db457-istiod-ca-cert\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:42:37.714323 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.714224 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-credential-socket\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:42:37.714731 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.714338 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "36148965-e677-471b-986e-f5b07d5db457" (UID: "36148965-e677-471b-986e-f5b07d5db457"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:42:37.714731 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.714604 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-istio-data" (OuterVolumeSpecName: "istio-data") pod "36148965-e677-471b-986e-f5b07d5db457" (UID: "36148965-e677-471b-986e-f5b07d5db457"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:42:37.716247 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.716215 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36148965-e677-471b-986e-f5b07d5db457-istio-token" (OuterVolumeSpecName: "istio-token") pod "36148965-e677-471b-986e-f5b07d5db457" (UID: "36148965-e677-471b-986e-f5b07d5db457"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:42:37.716355 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.716247 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "36148965-e677-471b-986e-f5b07d5db457" (UID: "36148965-e677-471b-986e-f5b07d5db457"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:42:37.716355 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.716274 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/36148965-e677-471b-986e-f5b07d5db457-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "36148965-e677-471b-986e-f5b07d5db457" (UID: "36148965-e677-471b-986e-f5b07d5db457"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 12:42:37.716355 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.716328 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36148965-e677-471b-986e-f5b07d5db457-kube-api-access-8q8gg" (OuterVolumeSpecName: "kube-api-access-8q8gg") pod "36148965-e677-471b-986e-f5b07d5db457" (UID: "36148965-e677-471b-986e-f5b07d5db457"). InnerVolumeSpecName "kube-api-access-8q8gg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:42:37.815589 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.815554 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36148965-e677-471b-986e-f5b07d5db457-istio-token\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:42:37.815589 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.815582 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-istio-data\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:42:37.815589 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.815596 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8q8gg\" (UniqueName: \"kubernetes.io/projected/36148965-e677-471b-986e-f5b07d5db457-kube-api-access-8q8gg\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:42:37.815824 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.815608 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-workload-certs\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:42:37.815824 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.815620 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36148965-e677-471b-986e-f5b07d5db457-istio-podinfo\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:42:37.815824 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.815631 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36148965-e677-471b-986e-f5b07d5db457-istio-envoy\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:42:37.954475 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.954437 2578 generic.go:356] "Generic (PLEG): container finished" podID="36148965-e677-471b-986e-f5b07d5db457" containerID="c22066ce45d7be415ac16f2e923d9e4da0adeaf510a046464a90fbf35bb45c38" exitCode=0 Jul 16 12:42:37.954649 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.954536 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" Jul 16 12:42:37.954649 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.954568 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" event={"ID":"36148965-e677-471b-986e-f5b07d5db457","Type":"ContainerDied","Data":"c22066ce45d7be415ac16f2e923d9e4da0adeaf510a046464a90fbf35bb45c38"} Jul 16 12:42:37.954649 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.954622 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs" event={"ID":"36148965-e677-471b-986e-f5b07d5db457","Type":"ContainerDied","Data":"c5df35d20d793855536d35098ddc368573bc87eff17f6508ee77fd235de2901b"} Jul 16 12:42:37.954763 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.954643 2578 scope.go:117] "RemoveContainer" containerID="c22066ce45d7be415ac16f2e923d9e4da0adeaf510a046464a90fbf35bb45c38" Jul 16 12:42:37.964612 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.964594 2578 scope.go:117] "RemoveContainer" containerID="b9bfe87bb66ea852dd29d2beef34dcc43b8e0c9a29b33dfa01601a877d34ab6a" Jul 16 12:42:37.973192 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.973167 2578 scope.go:117] "RemoveContainer" containerID="c22066ce45d7be415ac16f2e923d9e4da0adeaf510a046464a90fbf35bb45c38" Jul 16 12:42:37.973510 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:42:37.973491 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c22066ce45d7be415ac16f2e923d9e4da0adeaf510a046464a90fbf35bb45c38\": container with ID starting with c22066ce45d7be415ac16f2e923d9e4da0adeaf510a046464a90fbf35bb45c38 not found: ID does not exist" containerID="c22066ce45d7be415ac16f2e923d9e4da0adeaf510a046464a90fbf35bb45c38" Jul 16 12:42:37.973562 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.973521 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c22066ce45d7be415ac16f2e923d9e4da0adeaf510a046464a90fbf35bb45c38"} err="failed to get container status \"c22066ce45d7be415ac16f2e923d9e4da0adeaf510a046464a90fbf35bb45c38\": rpc error: code = NotFound desc = could not find container \"c22066ce45d7be415ac16f2e923d9e4da0adeaf510a046464a90fbf35bb45c38\": container with ID starting with c22066ce45d7be415ac16f2e923d9e4da0adeaf510a046464a90fbf35bb45c38 not found: ID does not exist" Jul 16 12:42:37.973562 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.973541 2578 scope.go:117] "RemoveContainer" containerID="b9bfe87bb66ea852dd29d2beef34dcc43b8e0c9a29b33dfa01601a877d34ab6a" Jul 16 12:42:37.973804 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:42:37.973784 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9bfe87bb66ea852dd29d2beef34dcc43b8e0c9a29b33dfa01601a877d34ab6a\": container with ID starting with b9bfe87bb66ea852dd29d2beef34dcc43b8e0c9a29b33dfa01601a877d34ab6a not found: ID does not exist" containerID="b9bfe87bb66ea852dd29d2beef34dcc43b8e0c9a29b33dfa01601a877d34ab6a" Jul 16 12:42:37.973848 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.973811 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9bfe87bb66ea852dd29d2beef34dcc43b8e0c9a29b33dfa01601a877d34ab6a"} err="failed to get container status \"b9bfe87bb66ea852dd29d2beef34dcc43b8e0c9a29b33dfa01601a877d34ab6a\": rpc error: code = NotFound desc = could not find container \"b9bfe87bb66ea852dd29d2beef34dcc43b8e0c9a29b33dfa01601a877d34ab6a\": container with ID starting with b9bfe87bb66ea852dd29d2beef34dcc43b8e0c9a29b33dfa01601a877d34ab6a not found: ID does not exist" Jul 16 12:42:37.979908 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.979880 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs"] Jul 16 12:42:37.987333 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:37.987309 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-28d017-openshift-default-58b785b44f-g6gzs"] Jul 16 12:42:39.315230 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:39.315195 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:39.442969 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:39.442923 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:39.479045 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:39.479011 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36148965-e677-471b-986e-f5b07d5db457" path="/var/lib/kubelet/pods/36148965-e677-471b-986e-f5b07d5db457/volumes" Jul 16 12:42:39.625798 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:39.625706 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:39.912153 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:39.912055 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:40.303245 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.303211 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:40.625839 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.625761 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw"] Jul 16 12:42:40.626204 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.626124 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36148965-e677-471b-986e-f5b07d5db457" containerName="istio-proxy" Jul 16 12:42:40.626204 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.626134 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="36148965-e677-471b-986e-f5b07d5db457" containerName="istio-proxy" Jul 16 12:42:40.626204 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.626142 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36148965-e677-471b-986e-f5b07d5db457" containerName="istio-proxy" Jul 16 12:42:40.626204 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.626148 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="36148965-e677-471b-986e-f5b07d5db457" containerName="istio-proxy" Jul 16 12:42:40.626348 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.626208 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="36148965-e677-471b-986e-f5b07d5db457" containerName="istio-proxy" Jul 16 12:42:40.626348 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.626215 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="36148965-e677-471b-986e-f5b07d5db457" containerName="istio-proxy" Jul 16 12:42:40.629283 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.629264 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.631527 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.631510 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-70ba56-openshift-default-dockercfg-bqwqw\"" Jul 16 12:42:40.638570 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.638546 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-workload-socket\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.638699 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.638675 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-workload-certs\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.638776 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.638731 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-envoy\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.638776 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.638766 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-data\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.638917 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.638890 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-credential-socket\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.638997 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.638976 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-token\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.639066 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.639012 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm8dm\" (UniqueName: \"kubernetes.io/projected/f842c344-9b60-43e9-b38d-e0cdb533fc61-kube-api-access-rm8dm\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.639372 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.639158 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f842c344-9b60-43e9-b38d-e0cdb533fc61-istiod-ca-cert\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.639372 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.639231 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-podinfo\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.640999 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.640981 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw"] Jul 16 12:42:40.649438 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.649383 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:40.740275 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.740242 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-podinfo\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.740275 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.740280 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-workload-socket\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.740571 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.740297 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-workload-certs\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.740571 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.740322 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-envoy\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.740571 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.740436 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-data\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.740571 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.740485 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-credential-socket\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.740571 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.740545 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-token\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.740571 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.740565 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm8dm\" (UniqueName: \"kubernetes.io/projected/f842c344-9b60-43e9-b38d-e0cdb533fc61-kube-api-access-rm8dm\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.740876 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.740627 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f842c344-9b60-43e9-b38d-e0cdb533fc61-istiod-ca-cert\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.740876 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.740757 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-workload-certs\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.740876 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.740774 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-workload-socket\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.740876 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.740834 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-data\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.741087 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.741059 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-credential-socket\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.741377 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.741355 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f842c344-9b60-43e9-b38d-e0cdb533fc61-istiod-ca-cert\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.742660 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.742635 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-envoy\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.742899 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.742882 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-podinfo\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.748850 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.748829 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-token\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.749146 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.749130 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm8dm\" (UniqueName: \"kubernetes.io/projected/f842c344-9b60-43e9-b38d-e0cdb533fc61-kube-api-access-rm8dm\") pod \"e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:40.809194 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.809159 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:40.939810 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:40.939716 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:41.078925 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:41.078898 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw"] Jul 16 12:42:41.082957 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:42:41.082896 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf842c344_9b60_43e9_b38d_e0cdb533fc61.slice/crio-54f9627663e220a108ce7dade328a0f320deef407fce6364eaa7ed712b9982c7 WatchSource:0}: Error finding container 54f9627663e220a108ce7dade328a0f320deef407fce6364eaa7ed712b9982c7: Status 404 returned error can't find the container with id 54f9627663e220a108ce7dade328a0f320deef407fce6364eaa7ed712b9982c7 Jul 16 12:42:41.085020 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:41.084987 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:42:41.085128 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:41.085066 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:42:41.085128 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:41.085098 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:42:41.978854 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:41.978821 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" event={"ID":"f842c344-9b60-43e9-b38d-e0cdb533fc61","Type":"ContainerStarted","Data":"ef66951db46b888ee4f49aea2907471936cfad97d8eacf3e60fa7a49a0a44a4e"} Jul 16 12:42:41.979257 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:41.978885 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" event={"ID":"f842c344-9b60-43e9-b38d-e0cdb533fc61","Type":"ContainerStarted","Data":"54f9627663e220a108ce7dade328a0f320deef407fce6364eaa7ed712b9982c7"} Jul 16 12:42:42.000842 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:42.000785 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" podStartSLOduration=2.000767294 podStartE2EDuration="2.000767294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:42:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:42:41.999711528 +0000 UTC m=+2111.763080993" watchObservedRunningTime="2026-07-16 12:42:42.000767294 +0000 UTC m=+2111.764136757" Jul 16 12:42:42.940684 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:42.940644 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:42.946043 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:42.946016 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:42.984784 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:42.984750 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:42.985751 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:42.985733 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:43.442320 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:43.442291 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:43.548384 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:43.548349 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:43.707664 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:43.707577 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:43.809088 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:43.809048 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:47.232047 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:47.232009 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:42:48.687980 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:48.687937 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:49.588795 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.588754 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:49.598124 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.598096 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs"] Jul 16 12:42:49.602498 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.602481 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.604849 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.604827 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-47e820-openshift-default-dockercfg-s9t27\"" Jul 16 12:42:49.634793 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.634743 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs"] Jul 16 12:42:49.683579 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.683531 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:49.733437 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.732922 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-istio-envoy\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.733437 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.732980 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c36d3709-cdba-4eed-8895-6450820f4262-istio-podinfo\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.733437 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.733030 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c36d3709-cdba-4eed-8895-6450820f4262-istio-token\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.733437 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.733070 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-workload-socket\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.733437 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.733105 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-credential-socket\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.733437 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.733140 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-istio-data\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.733437 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.733179 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-workload-certs\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.733437 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.733225 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c36d3709-cdba-4eed-8895-6450820f4262-istiod-ca-cert\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.733437 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.733256 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnww9\" (UniqueName: \"kubernetes.io/projected/c36d3709-cdba-4eed-8895-6450820f4262-kube-api-access-vnww9\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.834561 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.834514 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-istio-envoy\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.834789 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.834573 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c36d3709-cdba-4eed-8895-6450820f4262-istio-podinfo\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.834789 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.834622 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c36d3709-cdba-4eed-8895-6450820f4262-istio-token\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.834789 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.834661 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-workload-socket\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.834789 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.834695 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-credential-socket\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.834789 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.834729 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-istio-data\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.834789 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.834770 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-workload-certs\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.835156 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.834813 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c36d3709-cdba-4eed-8895-6450820f4262-istiod-ca-cert\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.835156 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.834843 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnww9\" (UniqueName: \"kubernetes.io/projected/c36d3709-cdba-4eed-8895-6450820f4262-kube-api-access-vnww9\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.835960 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.835926 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-credential-socket\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.837913 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.836757 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-workload-socket\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.837913 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.836819 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-workload-certs\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.837913 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.837008 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-istio-data\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.837913 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.837468 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c36d3709-cdba-4eed-8895-6450820f4262-istiod-ca-cert\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.840766 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.840701 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-istio-envoy\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.840952 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.840929 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c36d3709-cdba-4eed-8895-6450820f4262-istio-podinfo\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.844787 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.844757 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c36d3709-cdba-4eed-8895-6450820f4262-istio-token\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.844975 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.844953 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnww9\" (UniqueName: \"kubernetes.io/projected/c36d3709-cdba-4eed-8895-6450820f4262-kube-api-access-vnww9\") pod \"e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:49.913393 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:49.913362 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:50.099989 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:50.099880 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:50.101703 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:50.101683 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs"] Jul 16 12:42:50.104388 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:42:50.104362 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc36d3709_cdba_4eed_8895_6450820f4262.slice/crio-9e160b58c7d1289f0ca0c9443f79b4868c8071fff9c1355aefc0babe8dc691cf WatchSource:0}: Error finding container 9e160b58c7d1289f0ca0c9443f79b4868c8071fff9c1355aefc0babe8dc691cf: Status 404 returned error can't find the container with id 9e160b58c7d1289f0ca0c9443f79b4868c8071fff9c1355aefc0babe8dc691cf Jul 16 12:42:50.106831 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:50.106803 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:42:50.106896 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:50.106863 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:42:50.106896 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:50.106889 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:42:51.056329 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:51.056291 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" event={"ID":"c36d3709-cdba-4eed-8895-6450820f4262","Type":"ContainerStarted","Data":"6cbd699b8c5873bb5b4d8bb88ba8fe6b212616b8674aaa87bb3bac25bff24fc6"} Jul 16 12:42:51.056814 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:51.056353 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" event={"ID":"c36d3709-cdba-4eed-8895-6450820f4262","Type":"ContainerStarted","Data":"9e160b58c7d1289f0ca0c9443f79b4868c8071fff9c1355aefc0babe8dc691cf"} Jul 16 12:42:51.082131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:51.082081 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" podStartSLOduration=2.082065443 podStartE2EDuration="2.082065443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:42:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:42:51.079287167 +0000 UTC m=+2120.842656629" watchObservedRunningTime="2026-07-16 12:42:51.082065443 +0000 UTC m=+2120.845434906" Jul 16 12:42:51.914572 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:51.914534 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:51.935037 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:51.934999 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:52.063749 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:52.063714 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:52.064928 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:52.064904 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:42:52.124348 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:52.124311 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:52.216837 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:52.216738 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:52.278842 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:52.278806 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:52.383274 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:52.383238 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:52.857484 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:52.857442 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:52.957518 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:52.957476 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:55.080834 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:55.080804 2578 generic.go:356] "Generic (PLEG): container finished" podID="f842c344-9b60-43e9-b38d-e0cdb533fc61" containerID="ef66951db46b888ee4f49aea2907471936cfad97d8eacf3e60fa7a49a0a44a4e" exitCode=0 Jul 16 12:42:55.081280 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:55.080877 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" event={"ID":"f842c344-9b60-43e9-b38d-e0cdb533fc61","Type":"ContainerDied","Data":"ef66951db46b888ee4f49aea2907471936cfad97d8eacf3e60fa7a49a0a44a4e"} Jul 16 12:42:55.081280 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:55.081268 2578 scope.go:117] "RemoveContainer" containerID="ef66951db46b888ee4f49aea2907471936cfad97d8eacf3e60fa7a49a0a44a4e" Jul 16 12:42:55.082286 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:55.082253 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:42:55.082616 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:55.082358 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:42:55.082616 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:55.082392 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:42:56.087947 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:56.087912 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" event={"ID":"f842c344-9b60-43e9-b38d-e0cdb533fc61","Type":"ContainerStarted","Data":"aa9ceb301622242b4d9d3a670f2f4093a24484365b59303cfabec2a7b1513457"} Jul 16 12:42:56.940675 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:56.940636 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:56.941893 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:56.941860 2578 patch_prober.go:28] interesting pod/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 16 12:42:56.942115 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:56.942088 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" podUID="f842c344-9b60-43e9-b38d-e0cdb533fc61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 16 12:42:57.696299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:57.696267 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:42:57.940802 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:57.940763 2578 patch_prober.go:28] interesting pod/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 16 12:42:57.941076 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:57.941045 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" podUID="f842c344-9b60-43e9-b38d-e0cdb533fc61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 16 12:42:57.985084 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:57.984987 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:42:58.940323 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:58.940284 2578 patch_prober.go:28] interesting pod/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 16 12:42:58.940951 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:58.940917 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" podUID="f842c344-9b60-43e9-b38d-e0cdb533fc61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 16 12:42:59.940930 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:59.940892 2578 patch_prober.go:28] interesting pod/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 16 12:42:59.941524 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:42:59.941488 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" podUID="f842c344-9b60-43e9-b38d-e0cdb533fc61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 16 12:43:00.944139 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:00.944061 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:43:00.945266 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:00.945248 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:43:03.930736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:03.930704 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:43:04.131298 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:04.131267 2578 generic.go:356] "Generic (PLEG): container finished" podID="c36d3709-cdba-4eed-8895-6450820f4262" containerID="6cbd699b8c5873bb5b4d8bb88ba8fe6b212616b8674aaa87bb3bac25bff24fc6" exitCode=0 Jul 16 12:43:04.131496 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:04.131344 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" event={"ID":"c36d3709-cdba-4eed-8895-6450820f4262","Type":"ContainerDied","Data":"6cbd699b8c5873bb5b4d8bb88ba8fe6b212616b8674aaa87bb3bac25bff24fc6"} Jul 16 12:43:04.131807 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:04.131792 2578 scope.go:117] "RemoveContainer" containerID="6cbd699b8c5873bb5b4d8bb88ba8fe6b212616b8674aaa87bb3bac25bff24fc6" Jul 16 12:43:04.132824 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:04.132798 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:43:04.132898 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:04.132882 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:43:04.132961 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:04.132947 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:43:04.884137 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:04.884105 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:04.985023 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:04.984985 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:05.138401 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:05.138315 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" event={"ID":"c36d3709-cdba-4eed-8895-6450820f4262","Type":"ContainerStarted","Data":"aaa3ff150422214f811ae13f737a1e17756a44d4880f461b00413fba53498d17"} Jul 16 12:43:05.914079 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:05.914045 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:43:05.914977 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:05.914951 2578 patch_prober.go:28] interesting pod/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 16 12:43:05.915080 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:05.915001 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" podUID="c36d3709-cdba-4eed-8895-6450820f4262" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 16 12:43:06.526376 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:06.526336 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:06.914540 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:06.914508 2578 patch_prober.go:28] interesting pod/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 16 12:43:06.914711 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:06.914560 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" podUID="c36d3709-cdba-4eed-8895-6450820f4262" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 16 12:43:07.063677 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:07.063639 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:43:07.914562 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:07.914528 2578 patch_prober.go:28] interesting pod/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 16 12:43:07.914942 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:07.914585 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" podUID="c36d3709-cdba-4eed-8895-6450820f4262" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 16 12:43:08.914655 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:08.914616 2578 patch_prober.go:28] interesting pod/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 16 12:43:08.915034 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:08.914676 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" podUID="c36d3709-cdba-4eed-8895-6450820f4262" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 16 12:43:09.918569 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:09.918540 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:43:09.919564 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:09.919537 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:43:13.065161 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:13.065122 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:13.647247 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:13.647214 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:13.746261 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:13.746226 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:18.822734 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:18.822690 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:18.928865 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:18.928820 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:19.116736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:19.116653 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw"] Jul 16 12:43:19.116908 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:19.116883 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" podUID="f842c344-9b60-43e9-b38d-e0cdb533fc61" containerName="istio-proxy" containerID="cri-o://aa9ceb301622242b4d9d3a670f2f4093a24484365b59303cfabec2a7b1513457" gracePeriod=30 Jul 16 12:43:19.135564 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:19.135519 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:19.390614 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:19.390499 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:19.955310 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:19.955279 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:22.825143 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:22.825064 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:43:24.244322 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.244297 2578 generic.go:356] "Generic (PLEG): container finished" podID="f842c344-9b60-43e9-b38d-e0cdb533fc61" containerID="aa9ceb301622242b4d9d3a670f2f4093a24484365b59303cfabec2a7b1513457" exitCode=0 Jul 16 12:43:24.244671 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.244379 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" event={"ID":"f842c344-9b60-43e9-b38d-e0cdb533fc61","Type":"ContainerDied","Data":"aa9ceb301622242b4d9d3a670f2f4093a24484365b59303cfabec2a7b1513457"} Jul 16 12:43:24.244671 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.244474 2578 scope.go:117] "RemoveContainer" containerID="ef66951db46b888ee4f49aea2907471936cfad97d8eacf3e60fa7a49a0a44a4e" Jul 16 12:43:24.376154 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.376129 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:43:24.466001 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.465928 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-envoy\") pod \"f842c344-9b60-43e9-b38d-e0cdb533fc61\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " Jul 16 12:43:24.466001 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.465972 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f842c344-9b60-43e9-b38d-e0cdb533fc61-istiod-ca-cert\") pod \"f842c344-9b60-43e9-b38d-e0cdb533fc61\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " Jul 16 12:43:24.466268 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.466014 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-workload-certs\") pod \"f842c344-9b60-43e9-b38d-e0cdb533fc61\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " Jul 16 12:43:24.466268 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.466050 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-data\") pod \"f842c344-9b60-43e9-b38d-e0cdb533fc61\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " Jul 16 12:43:24.466268 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.466071 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-credential-socket\") pod \"f842c344-9b60-43e9-b38d-e0cdb533fc61\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " Jul 16 12:43:24.466268 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.466115 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-workload-socket\") pod \"f842c344-9b60-43e9-b38d-e0cdb533fc61\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " Jul 16 12:43:24.466268 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.466149 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-token\") pod \"f842c344-9b60-43e9-b38d-e0cdb533fc61\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " Jul 16 12:43:24.466268 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.466175 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rm8dm\" (UniqueName: \"kubernetes.io/projected/f842c344-9b60-43e9-b38d-e0cdb533fc61-kube-api-access-rm8dm\") pod \"f842c344-9b60-43e9-b38d-e0cdb533fc61\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " Jul 16 12:43:24.466268 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.466202 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-podinfo\") pod \"f842c344-9b60-43e9-b38d-e0cdb533fc61\" (UID: \"f842c344-9b60-43e9-b38d-e0cdb533fc61\") " Jul 16 12:43:24.466683 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.466338 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f842c344-9b60-43e9-b38d-e0cdb533fc61" (UID: "f842c344-9b60-43e9-b38d-e0cdb533fc61"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:43:24.466683 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.466361 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f842c344-9b60-43e9-b38d-e0cdb533fc61" (UID: "f842c344-9b60-43e9-b38d-e0cdb533fc61"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:43:24.466683 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.466456 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f842c344-9b60-43e9-b38d-e0cdb533fc61-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f842c344-9b60-43e9-b38d-e0cdb533fc61" (UID: "f842c344-9b60-43e9-b38d-e0cdb533fc61"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:43:24.466683 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.466610 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f842c344-9b60-43e9-b38d-e0cdb533fc61-istiod-ca-cert\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:24.466683 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.466632 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-workload-certs\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:24.466683 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.466650 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-credential-socket\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:24.467018 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.466760 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f842c344-9b60-43e9-b38d-e0cdb533fc61" (UID: "f842c344-9b60-43e9-b38d-e0cdb533fc61"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:43:24.467077 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.467031 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-data" (OuterVolumeSpecName: "istio-data") pod "f842c344-9b60-43e9-b38d-e0cdb533fc61" (UID: "f842c344-9b60-43e9-b38d-e0cdb533fc61"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:43:24.468273 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.468247 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-token" (OuterVolumeSpecName: "istio-token") pod "f842c344-9b60-43e9-b38d-e0cdb533fc61" (UID: "f842c344-9b60-43e9-b38d-e0cdb533fc61"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:43:24.468375 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.468273 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f842c344-9b60-43e9-b38d-e0cdb533fc61" (UID: "f842c344-9b60-43e9-b38d-e0cdb533fc61"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:43:24.468375 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.468337 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f842c344-9b60-43e9-b38d-e0cdb533fc61-kube-api-access-rm8dm" (OuterVolumeSpecName: "kube-api-access-rm8dm") pod "f842c344-9b60-43e9-b38d-e0cdb533fc61" (UID: "f842c344-9b60-43e9-b38d-e0cdb533fc61"). InnerVolumeSpecName "kube-api-access-rm8dm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:43:24.468479 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.468460 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f842c344-9b60-43e9-b38d-e0cdb533fc61" (UID: "f842c344-9b60-43e9-b38d-e0cdb533fc61"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 12:43:24.567380 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.567348 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-workload-socket\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:24.567380 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.567377 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-token\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:24.567610 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.567390 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rm8dm\" (UniqueName: \"kubernetes.io/projected/f842c344-9b60-43e9-b38d-e0cdb533fc61-kube-api-access-rm8dm\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:24.567610 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.567403 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-podinfo\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:24.567610 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.567442 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-envoy\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:24.567610 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:24.567453 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f842c344-9b60-43e9-b38d-e0cdb533fc61-istio-data\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:25.250812 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:25.250780 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" Jul 16 12:43:25.250812 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:25.250802 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw" event={"ID":"f842c344-9b60-43e9-b38d-e0cdb533fc61","Type":"ContainerDied","Data":"54f9627663e220a108ce7dade328a0f320deef407fce6364eaa7ed712b9982c7"} Jul 16 12:43:25.251272 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:25.250897 2578 scope.go:117] "RemoveContainer" containerID="aa9ceb301622242b4d9d3a670f2f4093a24484365b59303cfabec2a7b1513457" Jul 16 12:43:25.273097 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:25.273073 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw"] Jul 16 12:43:25.277739 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:25.277717 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-70ba56-openshift-default-7f68f4d9d9-jk7pw"] Jul 16 12:43:25.479207 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:25.479170 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f842c344-9b60-43e9-b38d-e0cdb533fc61" path="/var/lib/kubelet/pods/f842c344-9b60-43e9-b38d-e0cdb533fc61/volumes" Jul 16 12:43:25.961968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:25.961934 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:31.892015 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:31.891978 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:31.993556 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:31.993520 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:32.227189 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:32.227158 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs"] Jul 16 12:43:32.227390 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:32.227368 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" podUID="c36d3709-cdba-4eed-8895-6450820f4262" containerName="istio-proxy" containerID="cri-o://aaa3ff150422214f811ae13f737a1e17756a44d4880f461b00413fba53498d17" gracePeriod=30 Jul 16 12:43:32.230432 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:32.230392 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:32.500912 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:32.500825 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:32.922920 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:32.922889 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:33.245910 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.245836 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx"] Jul 16 12:43:33.246211 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.246198 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f842c344-9b60-43e9-b38d-e0cdb533fc61" containerName="istio-proxy" Jul 16 12:43:33.246211 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.246213 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f842c344-9b60-43e9-b38d-e0cdb533fc61" containerName="istio-proxy" Jul 16 12:43:33.246312 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.246293 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f842c344-9b60-43e9-b38d-e0cdb533fc61" containerName="istio-proxy" Jul 16 12:43:33.246312 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.246303 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f842c344-9b60-43e9-b38d-e0cdb533fc61" containerName="istio-proxy" Jul 16 12:43:33.250113 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.250096 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.252524 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.252502 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-cf65e4-openshift-default-dockercfg-gl8pj\"" Jul 16 12:43:33.266800 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.266779 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx"] Jul 16 12:43:33.273048 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.273025 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:33.344063 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.344028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-credential-socket\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.344214 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.344074 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-workload-certs\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.344214 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.344141 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-data\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.344214 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.344179 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-workload-socket\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.344369 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.344230 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85pd2\" (UniqueName: \"kubernetes.io/projected/9d53a7c2-97e8-4278-b1db-93c0e592e186-kube-api-access-85pd2\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.344369 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.344297 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-podinfo\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.344369 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.344338 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-envoy\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.344496 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.344377 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-token\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.344496 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.344485 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d53a7c2-97e8-4278-b1db-93c0e592e186-istiod-ca-cert\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.445322 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.445295 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-data\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.445507 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.445325 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-workload-socket\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.445507 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.445361 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85pd2\" (UniqueName: \"kubernetes.io/projected/9d53a7c2-97e8-4278-b1db-93c0e592e186-kube-api-access-85pd2\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.445507 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.445393 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-podinfo\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.445507 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.445433 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-envoy\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.445507 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.445463 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-token\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.445507 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.445504 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d53a7c2-97e8-4278-b1db-93c0e592e186-istiod-ca-cert\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.445843 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.445533 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-credential-socket\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.445843 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.445553 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-workload-certs\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.445843 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.445826 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-workload-socket\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.446004 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.445927 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-workload-certs\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.446004 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.445961 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-credential-socket\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.446101 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.446027 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-data\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.446263 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.446237 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d53a7c2-97e8-4278-b1db-93c0e592e186-istiod-ca-cert\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.447818 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.447801 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-envoy\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.448020 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.448004 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-podinfo\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.455862 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.455839 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85pd2\" (UniqueName: \"kubernetes.io/projected/9d53a7c2-97e8-4278-b1db-93c0e592e186-kube-api-access-85pd2\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.456006 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.455988 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-token\") pod \"e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.523529 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.523455 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:33.560569 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.560545 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:33.698482 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.698455 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx"] Jul 16 12:43:33.699343 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:43:33.699325 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d53a7c2_97e8_4278_b1db_93c0e592e186.slice/crio-789e11523407bcfbcbde67f84fd3207399186356ce04f85fb28ac5638b71c6a9 WatchSource:0}: Error finding container 789e11523407bcfbcbde67f84fd3207399186356ce04f85fb28ac5638b71c6a9: Status 404 returned error can't find the container with id 789e11523407bcfbcbde67f84fd3207399186356ce04f85fb28ac5638b71c6a9 Jul 16 12:43:33.701522 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.701496 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:43:33.701589 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.701556 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:43:33.701589 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:33.701584 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:43:34.300529 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:34.300494 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" event={"ID":"9d53a7c2-97e8-4278-b1db-93c0e592e186","Type":"ContainerStarted","Data":"64da36fb39c24e0f7974419953a52000d86c69d227fd85b5f5a7157cec485678"} Jul 16 12:43:34.301009 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:34.300544 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" event={"ID":"9d53a7c2-97e8-4278-b1db-93c0e592e186","Type":"ContainerStarted","Data":"789e11523407bcfbcbde67f84fd3207399186356ce04f85fb28ac5638b71c6a9"} Jul 16 12:43:34.322382 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:34.322339 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" podStartSLOduration=1.322323735 podStartE2EDuration="1.322323735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:43:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:43:34.319864469 +0000 UTC m=+2164.083233932" watchObservedRunningTime="2026-07-16 12:43:34.322323735 +0000 UTC m=+2164.085693197" Jul 16 12:43:34.561730 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:34.561654 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:34.566531 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:34.566508 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:35.305968 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:35.305932 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:35.306794 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:35.306777 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:35.384336 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:35.384287 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:35.476861 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:35.476826 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:35.537728 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:35.537677 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:35.638261 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:35.638160 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:36.127717 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:36.127674 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:36.228800 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:36.228763 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:37.481320 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.481297 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:43:37.589951 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.589921 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c36d3709-cdba-4eed-8895-6450820f4262-istiod-ca-cert\") pod \"c36d3709-cdba-4eed-8895-6450820f4262\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " Jul 16 12:43:37.590142 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.589959 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-istio-data\") pod \"c36d3709-cdba-4eed-8895-6450820f4262\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " Jul 16 12:43:37.590142 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.589981 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnww9\" (UniqueName: \"kubernetes.io/projected/c36d3709-cdba-4eed-8895-6450820f4262-kube-api-access-vnww9\") pod \"c36d3709-cdba-4eed-8895-6450820f4262\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " Jul 16 12:43:37.590142 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.590010 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c36d3709-cdba-4eed-8895-6450820f4262-istio-token\") pod \"c36d3709-cdba-4eed-8895-6450820f4262\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " Jul 16 12:43:37.590309 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.590187 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-workload-certs\") pod \"c36d3709-cdba-4eed-8895-6450820f4262\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " Jul 16 12:43:37.590309 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.590245 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-credential-socket\") pod \"c36d3709-cdba-4eed-8895-6450820f4262\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " Jul 16 12:43:37.590309 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.590281 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c36d3709-cdba-4eed-8895-6450820f4262-istio-podinfo\") pod \"c36d3709-cdba-4eed-8895-6450820f4262\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " Jul 16 12:43:37.590309 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.590303 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-workload-socket\") pod \"c36d3709-cdba-4eed-8895-6450820f4262\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " Jul 16 12:43:37.590558 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.590344 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-istio-envoy\") pod \"c36d3709-cdba-4eed-8895-6450820f4262\" (UID: \"c36d3709-cdba-4eed-8895-6450820f4262\") " Jul 16 12:43:37.590558 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.590370 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c36d3709-cdba-4eed-8895-6450820f4262-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c36d3709-cdba-4eed-8895-6450820f4262" (UID: "c36d3709-cdba-4eed-8895-6450820f4262"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:43:37.590558 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.590505 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c36d3709-cdba-4eed-8895-6450820f4262" (UID: "c36d3709-cdba-4eed-8895-6450820f4262"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:43:37.590720 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.590695 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c36d3709-cdba-4eed-8895-6450820f4262" (UID: "c36d3709-cdba-4eed-8895-6450820f4262"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:43:37.590783 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.590743 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c36d3709-cdba-4eed-8895-6450820f4262-istiod-ca-cert\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:37.590783 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.590760 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-workload-certs\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:37.590888 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.590779 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-istio-data" (OuterVolumeSpecName: "istio-data") pod "c36d3709-cdba-4eed-8895-6450820f4262" (UID: "c36d3709-cdba-4eed-8895-6450820f4262"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:43:37.590888 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.590799 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c36d3709-cdba-4eed-8895-6450820f4262" (UID: "c36d3709-cdba-4eed-8895-6450820f4262"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:43:37.592359 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.592335 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c36d3709-cdba-4eed-8895-6450820f4262-istio-token" (OuterVolumeSpecName: "istio-token") pod "c36d3709-cdba-4eed-8895-6450820f4262" (UID: "c36d3709-cdba-4eed-8895-6450820f4262"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:43:37.592480 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.592371 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c36d3709-cdba-4eed-8895-6450820f4262-kube-api-access-vnww9" (OuterVolumeSpecName: "kube-api-access-vnww9") pod "c36d3709-cdba-4eed-8895-6450820f4262" (UID: "c36d3709-cdba-4eed-8895-6450820f4262"). InnerVolumeSpecName "kube-api-access-vnww9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:43:37.592480 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.592458 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c36d3709-cdba-4eed-8895-6450820f4262-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c36d3709-cdba-4eed-8895-6450820f4262" (UID: "c36d3709-cdba-4eed-8895-6450820f4262"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 12:43:37.592598 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.592584 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c36d3709-cdba-4eed-8895-6450820f4262" (UID: "c36d3709-cdba-4eed-8895-6450820f4262"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:43:37.691228 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.691199 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-credential-socket\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:37.691228 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.691224 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c36d3709-cdba-4eed-8895-6450820f4262-istio-podinfo\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:37.691228 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.691236 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-workload-socket\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:37.691476 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.691245 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-istio-envoy\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:37.691476 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.691253 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c36d3709-cdba-4eed-8895-6450820f4262-istio-data\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:37.691476 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.691261 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnww9\" (UniqueName: \"kubernetes.io/projected/c36d3709-cdba-4eed-8895-6450820f4262-kube-api-access-vnww9\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:37.691476 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.691268 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c36d3709-cdba-4eed-8895-6450820f4262-istio-token\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:43:37.833106 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:37.833064 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:43:38.323675 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:38.323640 2578 generic.go:356] "Generic (PLEG): container finished" podID="c36d3709-cdba-4eed-8895-6450820f4262" containerID="aaa3ff150422214f811ae13f737a1e17756a44d4880f461b00413fba53498d17" exitCode=0 Jul 16 12:43:38.323675 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:38.323685 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" Jul 16 12:43:38.323893 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:38.323715 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" event={"ID":"c36d3709-cdba-4eed-8895-6450820f4262","Type":"ContainerDied","Data":"aaa3ff150422214f811ae13f737a1e17756a44d4880f461b00413fba53498d17"} Jul 16 12:43:38.323893 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:38.323761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs" event={"ID":"c36d3709-cdba-4eed-8895-6450820f4262","Type":"ContainerDied","Data":"9e160b58c7d1289f0ca0c9443f79b4868c8071fff9c1355aefc0babe8dc691cf"} Jul 16 12:43:38.323893 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:38.323796 2578 scope.go:117] "RemoveContainer" containerID="aaa3ff150422214f811ae13f737a1e17756a44d4880f461b00413fba53498d17" Jul 16 12:43:38.333856 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:38.333840 2578 scope.go:117] "RemoveContainer" containerID="6cbd699b8c5873bb5b4d8bb88ba8fe6b212616b8674aaa87bb3bac25bff24fc6" Jul 16 12:43:38.341751 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:38.341736 2578 scope.go:117] "RemoveContainer" containerID="aaa3ff150422214f811ae13f737a1e17756a44d4880f461b00413fba53498d17" Jul 16 12:43:38.341995 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:43:38.341975 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaa3ff150422214f811ae13f737a1e17756a44d4880f461b00413fba53498d17\": container with ID starting with aaa3ff150422214f811ae13f737a1e17756a44d4880f461b00413fba53498d17 not found: ID does not exist" containerID="aaa3ff150422214f811ae13f737a1e17756a44d4880f461b00413fba53498d17" Jul 16 12:43:38.342041 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:38.342003 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaa3ff150422214f811ae13f737a1e17756a44d4880f461b00413fba53498d17"} err="failed to get container status \"aaa3ff150422214f811ae13f737a1e17756a44d4880f461b00413fba53498d17\": rpc error: code = NotFound desc = could not find container \"aaa3ff150422214f811ae13f737a1e17756a44d4880f461b00413fba53498d17\": container with ID starting with aaa3ff150422214f811ae13f737a1e17756a44d4880f461b00413fba53498d17 not found: ID does not exist" Jul 16 12:43:38.342041 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:38.342019 2578 scope.go:117] "RemoveContainer" containerID="6cbd699b8c5873bb5b4d8bb88ba8fe6b212616b8674aaa87bb3bac25bff24fc6" Jul 16 12:43:38.342238 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:43:38.342219 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cbd699b8c5873bb5b4d8bb88ba8fe6b212616b8674aaa87bb3bac25bff24fc6\": container with ID starting with 6cbd699b8c5873bb5b4d8bb88ba8fe6b212616b8674aaa87bb3bac25bff24fc6 not found: ID does not exist" containerID="6cbd699b8c5873bb5b4d8bb88ba8fe6b212616b8674aaa87bb3bac25bff24fc6" Jul 16 12:43:38.342297 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:38.342247 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cbd699b8c5873bb5b4d8bb88ba8fe6b212616b8674aaa87bb3bac25bff24fc6"} err="failed to get container status \"6cbd699b8c5873bb5b4d8bb88ba8fe6b212616b8674aaa87bb3bac25bff24fc6\": rpc error: code = NotFound desc = could not find container \"6cbd699b8c5873bb5b4d8bb88ba8fe6b212616b8674aaa87bb3bac25bff24fc6\": container with ID starting with 6cbd699b8c5873bb5b4d8bb88ba8fe6b212616b8674aaa87bb3bac25bff24fc6 not found: ID does not exist" Jul 16 12:43:38.349229 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:38.349210 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs"] Jul 16 12:43:38.353487 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:38.353464 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-47e820-openshift-default-58bcc859f7-v5sbs"] Jul 16 12:43:39.479383 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:39.479339 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c36d3709-cdba-4eed-8895-6450820f4262" path="/var/lib/kubelet/pods/c36d3709-cdba-4eed-8895-6450820f4262/volumes" Jul 16 12:43:40.942830 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:40.942794 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:41.331290 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.331257 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:41.645619 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.645528 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8"] Jul 16 12:43:41.645981 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.645962 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c36d3709-cdba-4eed-8895-6450820f4262" containerName="istio-proxy" Jul 16 12:43:41.646068 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.645984 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c36d3709-cdba-4eed-8895-6450820f4262" containerName="istio-proxy" Jul 16 12:43:41.646068 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.646001 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f842c344-9b60-43e9-b38d-e0cdb533fc61" containerName="istio-proxy" Jul 16 12:43:41.646068 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.646009 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f842c344-9b60-43e9-b38d-e0cdb533fc61" containerName="istio-proxy" Jul 16 12:43:41.646068 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.646029 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c36d3709-cdba-4eed-8895-6450820f4262" containerName="istio-proxy" Jul 16 12:43:41.646068 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.646038 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c36d3709-cdba-4eed-8895-6450820f4262" containerName="istio-proxy" Jul 16 12:43:41.646304 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.646132 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c36d3709-cdba-4eed-8895-6450820f4262" containerName="istio-proxy" Jul 16 12:43:41.646304 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.646150 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c36d3709-cdba-4eed-8895-6450820f4262" containerName="istio-proxy" Jul 16 12:43:41.649148 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.649126 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.651852 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.651829 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-3c72a3-openshift-default-dockercfg-lfpr8\"" Jul 16 12:43:41.665954 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.665927 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8"] Jul 16 12:43:41.681877 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.681850 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:41.727535 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.727513 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-envoy\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.727657 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.727541 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-data\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.727657 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.727578 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-credential-socket\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.727657 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.727606 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzdqn\" (UniqueName: \"kubernetes.io/projected/6a498dfe-73c9-45dd-8456-b9012fc0254f-kube-api-access-zzdqn\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.727657 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.727626 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-workload-certs\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.727786 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.727698 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-workload-socket\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.727786 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.727724 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-podinfo\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.727873 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.727803 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-token\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.727873 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.727832 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a498dfe-73c9-45dd-8456-b9012fc0254f-istiod-ca-cert\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.828289 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.828263 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-envoy\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.828500 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.828295 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-data\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.828500 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.828328 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-credential-socket\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.828500 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.828445 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzdqn\" (UniqueName: \"kubernetes.io/projected/6a498dfe-73c9-45dd-8456-b9012fc0254f-kube-api-access-zzdqn\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.828500 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.828479 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-workload-certs\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.828718 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.828532 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-workload-socket\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.828718 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.828548 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-podinfo\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.828718 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.828618 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-token\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.828718 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.828656 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a498dfe-73c9-45dd-8456-b9012fc0254f-istiod-ca-cert\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.828908 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.828760 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-data\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.828908 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.828766 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-credential-socket\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.828908 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.828856 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-workload-certs\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.829122 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.829097 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-workload-socket\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.829316 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.829299 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a498dfe-73c9-45dd-8456-b9012fc0254f-istiod-ca-cert\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.830595 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.830575 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-envoy\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.830962 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.830943 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-podinfo\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.840161 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.840137 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-token\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.840469 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.840450 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzdqn\" (UniqueName: \"kubernetes.io/projected/6a498dfe-73c9-45dd-8456-b9012fc0254f-kube-api-access-zzdqn\") pod \"e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:41.958981 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:41.958888 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:42.112387 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:42.112359 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8"] Jul 16 12:43:42.113466 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:43:42.113436 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a498dfe_73c9_45dd_8456_b9012fc0254f.slice/crio-73be57e58784a07841bd7e581e9ef798a84c2db8999d0751bb81e3032e924891 WatchSource:0}: Error finding container 73be57e58784a07841bd7e581e9ef798a84c2db8999d0751bb81e3032e924891: Status 404 returned error can't find the container with id 73be57e58784a07841bd7e581e9ef798a84c2db8999d0751bb81e3032e924891 Jul 16 12:43:42.115475 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:42.115447 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:43:42.115543 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:42.115506 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:43:42.115543 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:42.115531 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:43:42.257735 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:42.257639 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:42.349476 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:42.349436 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" event={"ID":"6a498dfe-73c9-45dd-8456-b9012fc0254f","Type":"ContainerStarted","Data":"57aa2c39f8845d8ef1490ab7a2aea02313f967dfbc705e1f24e10c30d107358b"} Jul 16 12:43:42.349635 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:42.349494 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" event={"ID":"6a498dfe-73c9-45dd-8456-b9012fc0254f","Type":"ContainerStarted","Data":"73be57e58784a07841bd7e581e9ef798a84c2db8999d0751bb81e3032e924891"} Jul 16 12:43:42.375077 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:42.375027 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" podStartSLOduration=1.375011911 podStartE2EDuration="1.375011911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:43:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:43:42.372042088 +0000 UTC m=+2172.135411563" watchObservedRunningTime="2026-07-16 12:43:42.375011911 +0000 UTC m=+2172.138381372" Jul 16 12:43:42.959946 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:42.959915 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:43.398646 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:43.398613 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:43.497131 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:43.497096 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:43.569949 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:43.569914 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:43.946110 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:43.946079 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:43.965715 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:43.965691 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:44.222919 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:44.222825 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:44.326548 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:44.326505 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:44.361180 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:44.361156 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:44.361959 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:44.361943 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:43:49.008375 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:49.008337 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:51.347572 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:51.347532 2578 patch_prober.go:28] interesting pod/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 16 12:43:51.347936 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:51.347616 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" podUID="9d53a7c2-97e8-4278-b1db-93c0e592e186" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 16 12:43:51.925597 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:51.925559 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:43:53.417489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:53.417450 2578 generic.go:356] "Generic (PLEG): container finished" podID="9d53a7c2-97e8-4278-b1db-93c0e592e186" containerID="64da36fb39c24e0f7974419953a52000d86c69d227fd85b5f5a7157cec485678" exitCode=0 Jul 16 12:43:53.418008 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:53.417501 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" event={"ID":"9d53a7c2-97e8-4278-b1db-93c0e592e186","Type":"ContainerDied","Data":"64da36fb39c24e0f7974419953a52000d86c69d227fd85b5f5a7157cec485678"} Jul 16 12:43:53.418074 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:53.418001 2578 scope.go:117] "RemoveContainer" containerID="64da36fb39c24e0f7974419953a52000d86c69d227fd85b5f5a7157cec485678" Jul 16 12:43:53.420026 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:53.419975 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:43:53.420563 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:53.420532 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:43:53.421144 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:53.421105 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:43:54.428120 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:54.428081 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" event={"ID":"9d53a7c2-97e8-4278-b1db-93c0e592e186","Type":"ContainerStarted","Data":"86f63d93a38bbcaa5e3076663cfa82a867701044653f33d7c265d8f0c30310d7"} Jul 16 12:43:54.561678 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:54.561646 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:54.561861 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:54.561839 2578 patch_prober.go:28] interesting pod/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 16 12:43:54.561911 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:54.561887 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" podUID="9d53a7c2-97e8-4278-b1db-93c0e592e186" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 16 12:43:55.561279 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:55.561233 2578 patch_prober.go:28] interesting pod/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 16 12:43:55.561807 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:55.561302 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" podUID="9d53a7c2-97e8-4278-b1db-93c0e592e186" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 16 12:43:56.561902 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:56.561861 2578 patch_prober.go:28] interesting pod/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 16 12:43:56.562533 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:56.562498 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" podUID="9d53a7c2-97e8-4278-b1db-93c0e592e186" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 16 12:43:56.962626 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:56.962539 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:57.064127 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:57.064087 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:43:57.561141 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:57.561098 2578 patch_prober.go:28] interesting pod/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 16 12:43:57.561349 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:57.561166 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" podUID="9d53a7c2-97e8-4278-b1db-93c0e592e186" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 16 12:43:58.561021 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:58.560990 2578 patch_prober.go:28] interesting pod/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 16 12:43:58.561399 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:58.561040 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" podUID="9d53a7c2-97e8-4278-b1db-93c0e592e186" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 16 12:43:59.565110 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:59.565078 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:59.565571 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:59.565445 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:43:59.566108 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:43:59.566088 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:44:00.112657 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:00.112218 2578 patch_prober.go:28] interesting pod/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": read tcp 10.134.0.2:36526->10.134.0.46:15021: read: connection reset by peer" start-of-body= Jul 16 12:44:00.112657 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:00.112294 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" podUID="6a498dfe-73c9-45dd-8456-b9012fc0254f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": read tcp 10.134.0.2:36526->10.134.0.46:15021: read: connection reset by peer" Jul 16 12:44:00.471057 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:00.470976 2578 generic.go:356] "Generic (PLEG): container finished" podID="6a498dfe-73c9-45dd-8456-b9012fc0254f" containerID="57aa2c39f8845d8ef1490ab7a2aea02313f967dfbc705e1f24e10c30d107358b" exitCode=0 Jul 16 12:44:00.471057 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:00.471047 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" event={"ID":"6a498dfe-73c9-45dd-8456-b9012fc0254f","Type":"ContainerDied","Data":"57aa2c39f8845d8ef1490ab7a2aea02313f967dfbc705e1f24e10c30d107358b"} Jul 16 12:44:00.471581 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:00.471568 2578 scope.go:117] "RemoveContainer" containerID="57aa2c39f8845d8ef1490ab7a2aea02313f967dfbc705e1f24e10c30d107358b" Jul 16 12:44:00.472531 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:00.472504 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:44:00.472648 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:00.472579 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:44:00.472648 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:00.472625 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 12:44:01.481055 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:01.481008 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" event={"ID":"6a498dfe-73c9-45dd-8456-b9012fc0254f","Type":"ContainerStarted","Data":"b96a080da4a8904eb0e01415615b5fd403429769524c3e734d784d5ea492b062"} Jul 16 12:44:01.959007 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:01.958975 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:44:01.959300 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:01.959282 2578 patch_prober.go:28] interesting pod/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 16 12:44:01.959345 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:01.959324 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" podUID="6a498dfe-73c9-45dd-8456-b9012fc0254f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 16 12:44:02.959523 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:02.959484 2578 patch_prober.go:28] interesting pod/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 16 12:44:02.959931 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:02.959540 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" podUID="6a498dfe-73c9-45dd-8456-b9012fc0254f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 16 12:44:03.959308 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:03.959273 2578 patch_prober.go:28] interesting pod/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 16 12:44:03.959503 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:03.959331 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" podUID="6a498dfe-73c9-45dd-8456-b9012fc0254f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 16 12:44:04.959455 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:04.959396 2578 patch_prober.go:28] interesting pod/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 16 12:44:04.959918 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:04.959476 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" podUID="6a498dfe-73c9-45dd-8456-b9012fc0254f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 16 12:44:05.962657 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:05.962627 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:44:05.963110 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:05.963025 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:44:05.963673 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:05.963652 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:44:06.056956 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:06.056920 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:44:06.157366 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:06.157329 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:44:08.752733 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:08.752700 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:44:12.824152 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:12.824117 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:44:30.920438 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:30.920398 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:44:35.970610 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:35.970572 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:44:37.305913 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:37.305877 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:44:41.137388 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:41.137355 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:44:56.921087 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:44:56.921010 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:45:00.161949 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.161910 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-df8d3cf1-29736765-wpfs2"] Jul 16 12:45:00.164486 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.164465 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-15231c4d-29736765-7gc2f"] Jul 16 12:45:00.164622 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.164600 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-df8d3cf1-29736765-wpfs2" Jul 16 12:45:00.166576 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.166560 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c7727fcf-29736765-j59kd"] Jul 16 12:45:00.166690 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.166659 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-15231c4d-29736765-7gc2f" Jul 16 12:45:00.168629 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.168608 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28d017-29736765-8krjw"] Jul 16 12:45:00.169514 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.169493 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c7727fcf-29736765-j59kd" Jul 16 12:45:00.170075 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.170054 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vxbx5\"" Jul 16 12:45:00.171797 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.171778 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-051a7b7b-29736765-p5plb"] Jul 16 12:45:00.171955 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.171897 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28d017-29736765-8krjw" Jul 16 12:45:00.174043 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.174025 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-051a7b7b-29736765-p5plb" Jul 16 12:45:00.176921 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.176904 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-df8d3cf1-29736765-wpfs2" Jul 16 12:45:00.184150 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.184132 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-15231c4d-29736765-7gc2f" Jul 16 12:45:00.190033 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.190012 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-df8d3cf1-29736765-wpfs2"] Jul 16 12:45:00.195601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.195558 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-051a7b7b-29736765-p5plb"] Jul 16 12:45:00.199474 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.199453 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-15231c4d-29736765-7gc2f"] Jul 16 12:45:00.202190 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.202173 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-47e820-29736765-krvb2"] Jul 16 12:45:00.206175 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.206152 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-47e820-29736765-krvb2" Jul 16 12:45:00.207666 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.207615 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28d017-29736765-8krjw"] Jul 16 12:45:00.210696 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.210616 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-70ba56-29736765-7vssx"] Jul 16 12:45:00.214797 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.214651 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-70ba56-29736765-7vssx" Jul 16 12:45:00.215113 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.215090 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3c72a3-29736765-p9vjf"] Jul 16 12:45:00.219227 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.219203 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cf65e4-29736765-g74dp"] Jul 16 12:45:00.219330 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.219317 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3c72a3-29736765-p9vjf" Jul 16 12:45:00.224490 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.224432 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c7727fcf-29736765-j59kd" Jul 16 12:45:00.225696 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.225677 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cf65e4-29736765-g74dp" Jul 16 12:45:00.229509 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.228349 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-70ba56-29736765-7vssx"] Jul 16 12:45:00.236198 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.234854 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-47e820-29736765-krvb2"] Jul 16 12:45:00.236198 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.234906 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c7727fcf-29736765-j59kd"] Jul 16 12:45:00.241758 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.241718 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28d017-29736765-8krjw" Jul 16 12:45:00.254854 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.251775 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cf65e4-29736765-g74dp"] Jul 16 12:45:00.260092 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.260070 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3c72a3-29736765-p9vjf"] Jul 16 12:45:00.298148 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.297681 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-051a7b7b-29736765-p5plb" Jul 16 12:45:00.311334 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.311310 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-47e820-29736765-krvb2" Jul 16 12:45:00.335074 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.334889 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-70ba56-29736765-7vssx" Jul 16 12:45:00.353754 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.353730 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3c72a3-29736765-p9vjf" Jul 16 12:45:00.371059 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.370894 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cf65e4-29736765-g74dp" Jul 16 12:45:00.496578 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.496530 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-15231c4d-29736765-7gc2f"] Jul 16 12:45:00.507834 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.506946 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 12:45:00.605894 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.605869 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-df8d3cf1-29736765-wpfs2"] Jul 16 12:45:00.607606 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:45:00.607581 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod609b6a64_2004_4043_8c79_d34ba775b4c9.slice/crio-ccd17d696a2d4d650ee6122bf93038db2a206a2644897fa3c8ad6f9151524363 WatchSource:0}: Error finding container ccd17d696a2d4d650ee6122bf93038db2a206a2644897fa3c8ad6f9151524363: Status 404 returned error can't find the container with id ccd17d696a2d4d650ee6122bf93038db2a206a2644897fa3c8ad6f9151524363 Jul 16 12:45:00.737168 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.737147 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c7727fcf-29736765-j59kd"] Jul 16 12:45:00.739324 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:45:00.739300 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39d45b32_de65_4feb_97d5_7a89935919bb.slice/crio-ae6a7a96de0636da43bf05322ea16488eb39495877917e63e40595d18c722c44 WatchSource:0}: Error finding container ae6a7a96de0636da43bf05322ea16488eb39495877917e63e40595d18c722c44: Status 404 returned error can't find the container with id ae6a7a96de0636da43bf05322ea16488eb39495877917e63e40595d18c722c44 Jul 16 12:45:00.739916 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:45:00.739888 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97365f2c_f17c_4b52_adb5_6bdcf641dd6b.slice/crio-6e82b8f8a0bf185e925a67800bf0ed0019f103a93edac269a576a7b63e3be702 WatchSource:0}: Error finding container 6e82b8f8a0bf185e925a67800bf0ed0019f103a93edac269a576a7b63e3be702: Status 404 returned error can't find the container with id 6e82b8f8a0bf185e925a67800bf0ed0019f103a93edac269a576a7b63e3be702 Jul 16 12:45:00.740305 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.740283 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28d017-29736765-8krjw"] Jul 16 12:45:00.788975 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.788948 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-051a7b7b-29736765-p5plb"] Jul 16 12:45:00.801187 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.801162 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-47e820-29736765-krvb2"] Jul 16 12:45:00.803302 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.803274 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c7727fcf-29736765-j59kd" event={"ID":"39d45b32-de65-4feb-97d5-7a89935919bb","Type":"ContainerStarted","Data":"ae6a7a96de0636da43bf05322ea16488eb39495877917e63e40595d18c722c44"} Jul 16 12:45:00.804773 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.804748 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28d017-29736765-8krjw" event={"ID":"97365f2c-f17c-4b52-adb5-6bdcf641dd6b","Type":"ContainerStarted","Data":"6e82b8f8a0bf185e925a67800bf0ed0019f103a93edac269a576a7b63e3be702"} Jul 16 12:45:00.806263 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.806239 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-15231c4d-29736765-7gc2f" event={"ID":"bd12cbbf-e04b-48e1-8540-cb93e5e7ff40","Type":"ContainerStarted","Data":"5dc766f1a4d9bb7b57db527d990b70f16d3784398256c6fcba810e6e5d5af6bc"} Jul 16 12:45:00.807531 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.807511 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-df8d3cf1-29736765-wpfs2" event={"ID":"609b6a64-2004-4043-8c79-d34ba775b4c9","Type":"ContainerStarted","Data":"ccd17d696a2d4d650ee6122bf93038db2a206a2644897fa3c8ad6f9151524363"} Jul 16 12:45:00.830619 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.830577 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3c72a3-29736765-p9vjf"] Jul 16 12:45:00.832552 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:45:00.832528 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cd43e61_cc0a_4c2d_beef_06905c7b2486.slice/crio-2d453bd72e1622d1a1075d3a415d501ab7cacf6c47e90760a6886a885d97c360 WatchSource:0}: Error finding container 2d453bd72e1622d1a1075d3a415d501ab7cacf6c47e90760a6886a885d97c360: Status 404 returned error can't find the container with id 2d453bd72e1622d1a1075d3a415d501ab7cacf6c47e90760a6886a885d97c360 Jul 16 12:45:00.834107 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.834089 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-70ba56-29736765-7vssx"] Jul 16 12:45:00.835269 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:45:00.835237 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ec1322_aa86_4615_af84_16b7ddd82014.slice/crio-a9b330566d4c07756c62493b6b40511c2c58fb3b8b458e7c38f1c6f4bfcd64df WatchSource:0}: Error finding container a9b330566d4c07756c62493b6b40511c2c58fb3b8b458e7c38f1c6f4bfcd64df: Status 404 returned error can't find the container with id a9b330566d4c07756c62493b6b40511c2c58fb3b8b458e7c38f1c6f4bfcd64df Jul 16 12:45:00.836380 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:45:00.836361 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45a7c5e_adfc_4bc8_a904_25163dbbd9ed.slice/crio-b5dcac9ecd7f9ef6ad576c607014ccf98369611eb9233d59f6a2b651335b0647 WatchSource:0}: Error finding container b5dcac9ecd7f9ef6ad576c607014ccf98369611eb9233d59f6a2b651335b0647: Status 404 returned error can't find the container with id b5dcac9ecd7f9ef6ad576c607014ccf98369611eb9233d59f6a2b651335b0647 Jul 16 12:45:00.837167 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:00.837146 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cf65e4-29736765-g74dp"] Jul 16 12:45:01.829276 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:01.829239 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3c72a3-29736765-p9vjf" event={"ID":"0cd43e61-cc0a-4c2d-beef-06905c7b2486","Type":"ContainerStarted","Data":"2d453bd72e1622d1a1075d3a415d501ab7cacf6c47e90760a6886a885d97c360"} Jul 16 12:45:01.831318 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:01.831282 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-47e820-29736765-krvb2" event={"ID":"b11a7983-dd5e-469b-861b-7288e5e9495a","Type":"ContainerStarted","Data":"3252d7af0cae507d888a7331e8e2fc7d4d1c9f889fa2c038123c976f2f30fcc5"} Jul 16 12:45:01.833273 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:01.833246 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-70ba56-29736765-7vssx" event={"ID":"e45a7c5e-adfc-4bc8-a904-25163dbbd9ed","Type":"ContainerStarted","Data":"b5dcac9ecd7f9ef6ad576c607014ccf98369611eb9233d59f6a2b651335b0647"} Jul 16 12:45:01.835807 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:01.835779 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-051a7b7b-29736765-p5plb" event={"ID":"56f41862-a1ee-458f-9aff-e3801c36a42b","Type":"ContainerStarted","Data":"1e159c71e0d57f06c07d25e27dd8dc4dd6e28bbcf2e5e2e96debe89aed17ee07"} Jul 16 12:45:01.838141 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:01.838115 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cf65e4-29736765-g74dp" event={"ID":"71ec1322-aa86-4615-af84-16b7ddd82014","Type":"ContainerStarted","Data":"a9b330566d4c07756c62493b6b40511c2c58fb3b8b458e7c38f1c6f4bfcd64df"} Jul 16 12:45:04.873343 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:04.873312 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-df8d3cf1-29736765-wpfs2" event={"ID":"609b6a64-2004-4043-8c79-d34ba775b4c9","Type":"ContainerStarted","Data":"97662bc2e959b4e2b970e6a3a9b4d8e38297dfad6a745cc6d94c486c65da2f4e"} Jul 16 12:45:04.874827 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:04.874805 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c7727fcf-29736765-j59kd" event={"ID":"39d45b32-de65-4feb-97d5-7a89935919bb","Type":"ContainerStarted","Data":"3b213f4e1c9b751c2293f1f81b7a25461b90da0dc0549f2433a7f3df684e81b5"} Jul 16 12:45:04.876117 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:04.876097 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3c72a3-29736765-p9vjf" event={"ID":"0cd43e61-cc0a-4c2d-beef-06905c7b2486","Type":"ContainerStarted","Data":"61368063d05e840d9af6c0f0cbc8d4d01e4a1efca7a8461b1de55c56be679c11"} Jul 16 12:45:04.877505 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:04.877483 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-47e820-29736765-krvb2" event={"ID":"b11a7983-dd5e-469b-861b-7288e5e9495a","Type":"ContainerStarted","Data":"df4d09467d7be5991a5f9084a32fffafecc729cfb430bd42a28c16a03652f75c"} Jul 16 12:45:04.878867 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:04.878846 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-70ba56-29736765-7vssx" event={"ID":"e45a7c5e-adfc-4bc8-a904-25163dbbd9ed","Type":"ContainerStarted","Data":"e4d0b3ad7871f0593bb87163f1e349aa1d369224e131337c7476813eda43b412"} Jul 16 12:45:04.880304 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:04.880279 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-051a7b7b-29736765-p5plb" event={"ID":"56f41862-a1ee-458f-9aff-e3801c36a42b","Type":"ContainerStarted","Data":"988417b79c31ce6a2c4e03e46cf4bd3b7595cca5a58ea55eeeb1598185571e76"} Jul 16 12:45:04.881682 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:04.881662 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-15231c4d-29736765-7gc2f" event={"ID":"bd12cbbf-e04b-48e1-8540-cb93e5e7ff40","Type":"ContainerStarted","Data":"d5dc863ff207c276dcdd26300cdc4a82c551ff31f3814da2a7a552aa254930c1"} Jul 16 12:45:04.890783 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:04.890757 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28d017-29736765-8krjw" event={"ID":"97365f2c-f17c-4b52-adb5-6bdcf641dd6b","Type":"ContainerStarted","Data":"2736451b00b5b425c67efd0de993869b64e29e19c14479c226d29baddb3923c5"} Jul 16 12:45:04.892990 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:04.892935 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-df8d3cf1-29736765-wpfs2" podStartSLOduration=1.635689006 podStartE2EDuration="4.892911275s" totalImagesPullingTime="3.257222269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:45:04.891224123 +0000 UTC m=+2254.654593589" watchObservedRunningTime="2026-07-16 12:45:04.892911275 +0000 UTC m=+2254.656280740" Jul 16 12:45:04.893827 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:04.893806 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cf65e4-29736765-g74dp" event={"ID":"71ec1322-aa86-4615-af84-16b7ddd82014","Type":"ContainerStarted","Data":"8f92c9496801d4d1ffcafae7206d84b29c3fa17d71f6e4061e43e9adbd90de72"} Jul 16 12:45:04.911026 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:04.910985 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28d017-29736765-8krjw" podStartSLOduration=1.790441806 podStartE2EDuration="4.9109742s" totalImagesPullingTime="3.120532394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:45:04.908725929 +0000 UTC m=+2254.672095391" watchObservedRunningTime="2026-07-16 12:45:04.9109742 +0000 UTC m=+2254.674343661" Jul 16 12:45:04.925959 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:04.925917 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3c72a3-29736765-p9vjf" podStartSLOduration=1.807262888 podStartE2EDuration="4.925904103s" totalImagesPullingTime="3.118641215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:45:04.924878836 +0000 UTC m=+2254.688248298" watchObservedRunningTime="2026-07-16 12:45:04.925904103 +0000 UTC m=+2254.689273564" Jul 16 12:45:04.944764 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:04.944722 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-051a7b7b-29736765-p5plb" podStartSLOduration=1.883156124 podStartE2EDuration="4.944707776s" totalImagesPullingTime="3.061551652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:45:04.943176181 +0000 UTC m=+2254.706545645" watchObservedRunningTime="2026-07-16 12:45:04.944707776 +0000 UTC m=+2254.708077286" Jul 16 12:45:04.959537 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:04.959499 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-15231c4d-29736765-7gc2f" podStartSLOduration=1.014100211 podStartE2EDuration="4.959485564s" totalImagesPullingTime="3.945385353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:45:04.957364442 +0000 UTC m=+2254.720733904" watchObservedRunningTime="2026-07-16 12:45:04.959485564 +0000 UTC m=+2254.722855024" Jul 16 12:45:04.976179 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:04.976139 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-47e820-29736765-krvb2" podStartSLOduration=1.72187362 podStartE2EDuration="4.976127128s" totalImagesPullingTime="3.254253508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:45:04.97365898 +0000 UTC m=+2254.737028432" watchObservedRunningTime="2026-07-16 12:45:04.976127128 +0000 UTC m=+2254.739496589" Jul 16 12:45:04.992227 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:04.992158 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c7727fcf-29736765-j59kd" podStartSLOduration=1.481957205 podStartE2EDuration="4.99214648s" totalImagesPullingTime="3.510189275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:45:04.988802816 +0000 UTC m=+2254.752172278" watchObservedRunningTime="2026-07-16 12:45:04.99214648 +0000 UTC m=+2254.755516003" Jul 16 12:45:05.006733 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.006698 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-70ba56-29736765-7vssx" podStartSLOduration=1.489967845 podStartE2EDuration="5.00668754s" totalImagesPullingTime="3.516719695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:45:05.005155469 +0000 UTC m=+2254.768524933" watchObservedRunningTime="2026-07-16 12:45:05.00668754 +0000 UTC m=+2254.770057002" Jul 16 12:45:05.022903 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.022858 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cf65e4-29736765-g74dp" podStartSLOduration=1.954242484 podStartE2EDuration="5.022845649s" totalImagesPullingTime="3.068603165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:45:05.020249555 +0000 UTC m=+2254.783619018" watchObservedRunningTime="2026-07-16 12:45:05.022845649 +0000 UTC m=+2254.786215110" Jul 16 12:45:05.901935 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.901905 2578 generic.go:356] "Generic (PLEG): container finished" podID="609b6a64-2004-4043-8c79-d34ba775b4c9" containerID="97662bc2e959b4e2b970e6a3a9b4d8e38297dfad6a745cc6d94c486c65da2f4e" exitCode=0 Jul 16 12:45:05.902329 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.901981 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-df8d3cf1-29736765-wpfs2" event={"ID":"609b6a64-2004-4043-8c79-d34ba775b4c9","Type":"ContainerDied","Data":"97662bc2e959b4e2b970e6a3a9b4d8e38297dfad6a745cc6d94c486c65da2f4e"} Jul 16 12:45:05.903379 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.903356 2578 generic.go:356] "Generic (PLEG): container finished" podID="39d45b32-de65-4feb-97d5-7a89935919bb" containerID="3b213f4e1c9b751c2293f1f81b7a25461b90da0dc0549f2433a7f3df684e81b5" exitCode=0 Jul 16 12:45:05.903476 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.903400 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c7727fcf-29736765-j59kd" event={"ID":"39d45b32-de65-4feb-97d5-7a89935919bb","Type":"ContainerDied","Data":"3b213f4e1c9b751c2293f1f81b7a25461b90da0dc0549f2433a7f3df684e81b5"} Jul 16 12:45:05.904905 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.904886 2578 generic.go:356] "Generic (PLEG): container finished" podID="0cd43e61-cc0a-4c2d-beef-06905c7b2486" containerID="61368063d05e840d9af6c0f0cbc8d4d01e4a1efca7a8461b1de55c56be679c11" exitCode=0 Jul 16 12:45:05.904991 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.904949 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3c72a3-29736765-p9vjf" event={"ID":"0cd43e61-cc0a-4c2d-beef-06905c7b2486","Type":"ContainerDied","Data":"61368063d05e840d9af6c0f0cbc8d4d01e4a1efca7a8461b1de55c56be679c11"} Jul 16 12:45:05.906454 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.906424 2578 generic.go:356] "Generic (PLEG): container finished" podID="b11a7983-dd5e-469b-861b-7288e5e9495a" containerID="df4d09467d7be5991a5f9084a32fffafecc729cfb430bd42a28c16a03652f75c" exitCode=0 Jul 16 12:45:05.906555 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.906505 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-47e820-29736765-krvb2" event={"ID":"b11a7983-dd5e-469b-861b-7288e5e9495a","Type":"ContainerDied","Data":"df4d09467d7be5991a5f9084a32fffafecc729cfb430bd42a28c16a03652f75c"} Jul 16 12:45:05.907911 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.907883 2578 generic.go:356] "Generic (PLEG): container finished" podID="e45a7c5e-adfc-4bc8-a904-25163dbbd9ed" containerID="e4d0b3ad7871f0593bb87163f1e349aa1d369224e131337c7476813eda43b412" exitCode=0 Jul 16 12:45:05.907983 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.907944 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-70ba56-29736765-7vssx" event={"ID":"e45a7c5e-adfc-4bc8-a904-25163dbbd9ed","Type":"ContainerDied","Data":"e4d0b3ad7871f0593bb87163f1e349aa1d369224e131337c7476813eda43b412"} Jul 16 12:45:05.909242 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.909224 2578 generic.go:356] "Generic (PLEG): container finished" podID="56f41862-a1ee-458f-9aff-e3801c36a42b" containerID="988417b79c31ce6a2c4e03e46cf4bd3b7595cca5a58ea55eeeb1598185571e76" exitCode=0 Jul 16 12:45:05.909313 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.909300 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-051a7b7b-29736765-p5plb" event={"ID":"56f41862-a1ee-458f-9aff-e3801c36a42b","Type":"ContainerDied","Data":"988417b79c31ce6a2c4e03e46cf4bd3b7595cca5a58ea55eeeb1598185571e76"} Jul 16 12:45:05.910637 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.910616 2578 generic.go:356] "Generic (PLEG): container finished" podID="bd12cbbf-e04b-48e1-8540-cb93e5e7ff40" containerID="d5dc863ff207c276dcdd26300cdc4a82c551ff31f3814da2a7a552aa254930c1" exitCode=0 Jul 16 12:45:05.910720 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.910671 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-15231c4d-29736765-7gc2f" event={"ID":"bd12cbbf-e04b-48e1-8540-cb93e5e7ff40","Type":"ContainerDied","Data":"d5dc863ff207c276dcdd26300cdc4a82c551ff31f3814da2a7a552aa254930c1"} Jul 16 12:45:05.911949 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.911933 2578 generic.go:356] "Generic (PLEG): container finished" podID="97365f2c-f17c-4b52-adb5-6bdcf641dd6b" containerID="2736451b00b5b425c67efd0de993869b64e29e19c14479c226d29baddb3923c5" exitCode=0 Jul 16 12:45:05.912019 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.911989 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28d017-29736765-8krjw" event={"ID":"97365f2c-f17c-4b52-adb5-6bdcf641dd6b","Type":"ContainerDied","Data":"2736451b00b5b425c67efd0de993869b64e29e19c14479c226d29baddb3923c5"} Jul 16 12:45:05.913424 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.913395 2578 generic.go:356] "Generic (PLEG): container finished" podID="71ec1322-aa86-4615-af84-16b7ddd82014" containerID="8f92c9496801d4d1ffcafae7206d84b29c3fa17d71f6e4061e43e9adbd90de72" exitCode=0 Jul 16 12:45:05.913497 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:05.913443 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cf65e4-29736765-g74dp" event={"ID":"71ec1322-aa86-4615-af84-16b7ddd82014","Type":"ContainerDied","Data":"8f92c9496801d4d1ffcafae7206d84b29c3fa17d71f6e4061e43e9adbd90de72"} Jul 16 12:45:07.144558 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.144401 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c7727fcf-29736765-j59kd" Jul 16 12:45:07.229406 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.229272 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-47e820-29736765-krvb2" Jul 16 12:45:07.234229 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.234207 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28d017-29736765-8krjw" Jul 16 12:45:07.240834 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.240696 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-051a7b7b-29736765-p5plb" Jul 16 12:45:07.249123 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.249095 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3c72a3-29736765-p9vjf" Jul 16 12:45:07.263898 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.263874 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-70ba56-29736765-7vssx" Jul 16 12:45:07.267600 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.267578 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cf65e4-29736765-g74dp" Jul 16 12:45:07.271152 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.271132 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-15231c4d-29736765-7gc2f" Jul 16 12:45:07.275857 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.275830 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-df8d3cf1-29736765-wpfs2" Jul 16 12:45:07.926082 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.926058 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-15231c4d-29736765-7gc2f" Jul 16 12:45:07.926082 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.926066 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-15231c4d-29736765-7gc2f" event={"ID":"bd12cbbf-e04b-48e1-8540-cb93e5e7ff40","Type":"ContainerDied","Data":"5dc766f1a4d9bb7b57db527d990b70f16d3784398256c6fcba810e6e5d5af6bc"} Jul 16 12:45:07.926299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.926115 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5dc766f1a4d9bb7b57db527d990b70f16d3784398256c6fcba810e6e5d5af6bc" Jul 16 12:45:07.927331 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.927313 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28d017-29736765-8krjw" Jul 16 12:45:07.927331 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.927318 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28d017-29736765-8krjw" event={"ID":"97365f2c-f17c-4b52-adb5-6bdcf641dd6b","Type":"ContainerDied","Data":"6e82b8f8a0bf185e925a67800bf0ed0019f103a93edac269a576a7b63e3be702"} Jul 16 12:45:07.927539 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.927355 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e82b8f8a0bf185e925a67800bf0ed0019f103a93edac269a576a7b63e3be702" Jul 16 12:45:07.929389 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.929373 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cf65e4-29736765-g74dp" Jul 16 12:45:07.929503 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.929435 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cf65e4-29736765-g74dp" event={"ID":"71ec1322-aa86-4615-af84-16b7ddd82014","Type":"ContainerDied","Data":"a9b330566d4c07756c62493b6b40511c2c58fb3b8b458e7c38f1c6f4bfcd64df"} Jul 16 12:45:07.929503 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.929471 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9b330566d4c07756c62493b6b40511c2c58fb3b8b458e7c38f1c6f4bfcd64df" Jul 16 12:45:07.931295 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.931278 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-df8d3cf1-29736765-wpfs2" Jul 16 12:45:07.931295 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.931290 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-df8d3cf1-29736765-wpfs2" event={"ID":"609b6a64-2004-4043-8c79-d34ba775b4c9","Type":"ContainerDied","Data":"ccd17d696a2d4d650ee6122bf93038db2a206a2644897fa3c8ad6f9151524363"} Jul 16 12:45:07.931480 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.931323 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ccd17d696a2d4d650ee6122bf93038db2a206a2644897fa3c8ad6f9151524363" Jul 16 12:45:07.932769 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.932751 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c7727fcf-29736765-j59kd" event={"ID":"39d45b32-de65-4feb-97d5-7a89935919bb","Type":"ContainerDied","Data":"ae6a7a96de0636da43bf05322ea16488eb39495877917e63e40595d18c722c44"} Jul 16 12:45:07.932769 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.932762 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-c7727fcf-29736765-j59kd" Jul 16 12:45:07.932914 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.932787 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae6a7a96de0636da43bf05322ea16488eb39495877917e63e40595d18c722c44" Jul 16 12:45:07.934153 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.934134 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3c72a3-29736765-p9vjf" Jul 16 12:45:07.934239 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.934135 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-3c72a3-29736765-p9vjf" event={"ID":"0cd43e61-cc0a-4c2d-beef-06905c7b2486","Type":"ContainerDied","Data":"2d453bd72e1622d1a1075d3a415d501ab7cacf6c47e90760a6886a885d97c360"} Jul 16 12:45:07.934299 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.934248 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d453bd72e1622d1a1075d3a415d501ab7cacf6c47e90760a6886a885d97c360" Jul 16 12:45:07.935363 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.935348 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-47e820-29736765-krvb2" Jul 16 12:45:07.935472 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.935379 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-47e820-29736765-krvb2" event={"ID":"b11a7983-dd5e-469b-861b-7288e5e9495a","Type":"ContainerDied","Data":"3252d7af0cae507d888a7331e8e2fc7d4d1c9f889fa2c038123c976f2f30fcc5"} Jul 16 12:45:07.935472 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.935431 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3252d7af0cae507d888a7331e8e2fc7d4d1c9f889fa2c038123c976f2f30fcc5" Jul 16 12:45:07.936842 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.936828 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-70ba56-29736765-7vssx" Jul 16 12:45:07.936918 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.936843 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-70ba56-29736765-7vssx" event={"ID":"e45a7c5e-adfc-4bc8-a904-25163dbbd9ed","Type":"ContainerDied","Data":"b5dcac9ecd7f9ef6ad576c607014ccf98369611eb9233d59f6a2b651335b0647"} Jul 16 12:45:07.936918 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.936878 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5dcac9ecd7f9ef6ad576c607014ccf98369611eb9233d59f6a2b651335b0647" Jul 16 12:45:07.938294 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.938277 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-051a7b7b-29736765-p5plb" event={"ID":"56f41862-a1ee-458f-9aff-e3801c36a42b","Type":"ContainerDied","Data":"1e159c71e0d57f06c07d25e27dd8dc4dd6e28bbcf2e5e2e96debe89aed17ee07"} Jul 16 12:45:07.938377 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.938305 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e159c71e0d57f06c07d25e27dd8dc4dd6e28bbcf2e5e2e96debe89aed17ee07" Jul 16 12:45:07.938377 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:07.938312 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-051a7b7b-29736765-p5plb" Jul 16 12:45:12.628388 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:12.628354 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:45:34.030354 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:34.030318 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:45:53.145600 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:45:53.145570 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:46:15.029348 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:46:15.029312 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:46:32.228154 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:46:32.228072 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:46:50.749225 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:46:50.749196 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:46:51.626248 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:46:51.626212 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:46:56.444124 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:46:56.444087 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:47:06.426525 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:47:06.426474 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:47:13.028280 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:47:13.028236 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:47:27.407546 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:47:27.407492 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:47:30.904609 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:47:30.904579 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:47:30.913838 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:47:30.913800 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:47:30.915954 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:47:30.915930 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:47:30.921479 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:47:30.921459 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cx284_82627060-8f4c-4020-8e86-506df37a1e55/ovn-acl-logging/0.log" Jul 16 12:47:33.740021 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:47:33.739986 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:47:43.030636 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:47:43.030605 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:47:43.134557 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:47:43.134527 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:47:50.049755 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:47:50.049721 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:47:50.447433 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:47:50.447338 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:48:07.247802 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:48:07.247765 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:48:08.309075 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:48:08.309041 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:48:12.227569 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:48:12.227519 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:48:13.632720 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:48:13.632682 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:48:28.525594 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:48:28.525558 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:48:32.075604 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:48:32.075565 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:48:47.637300 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:48:47.637263 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:48:57.875266 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:48:57.875236 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:48:57.976595 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:48:57.976560 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:49:06.875015 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:06.874985 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:49:07.376007 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:07.375937 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:49:08.727188 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:08.727153 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:49:29.321445 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:29.321347 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:49:34.667231 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:34.667198 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:49:35.417364 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:35.417336 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:49:36.467973 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.467941 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7843-140115474801200"] Jul 16 12:49:36.468327 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468285 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cd43e61-cc0a-4c2d-beef-06905c7b2486" containerName="cleanup" Jul 16 12:49:36.468327 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468296 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cd43e61-cc0a-4c2d-beef-06905c7b2486" containerName="cleanup" Jul 16 12:49:36.468327 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468315 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97365f2c-f17c-4b52-adb5-6bdcf641dd6b" containerName="cleanup" Jul 16 12:49:36.468327 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468322 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="97365f2c-f17c-4b52-adb5-6bdcf641dd6b" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468330 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd12cbbf-e04b-48e1-8540-cb93e5e7ff40" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468335 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd12cbbf-e04b-48e1-8540-cb93e5e7ff40" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468343 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="609b6a64-2004-4043-8c79-d34ba775b4c9" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468348 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="609b6a64-2004-4043-8c79-d34ba775b4c9" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468355 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56f41862-a1ee-458f-9aff-e3801c36a42b" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468360 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="56f41862-a1ee-458f-9aff-e3801c36a42b" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468369 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39d45b32-de65-4feb-97d5-7a89935919bb" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468373 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="39d45b32-de65-4feb-97d5-7a89935919bb" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468381 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71ec1322-aa86-4615-af84-16b7ddd82014" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468386 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="71ec1322-aa86-4615-af84-16b7ddd82014" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468390 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e45a7c5e-adfc-4bc8-a904-25163dbbd9ed" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468395 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e45a7c5e-adfc-4bc8-a904-25163dbbd9ed" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468406 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b11a7983-dd5e-469b-861b-7288e5e9495a" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468424 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b11a7983-dd5e-469b-861b-7288e5e9495a" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468478 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="71ec1322-aa86-4615-af84-16b7ddd82014" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468485 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cd43e61-cc0a-4c2d-beef-06905c7b2486" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468490 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="97365f2c-f17c-4b52-adb5-6bdcf641dd6b" containerName="cleanup" Jul 16 12:49:36.468489 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468496 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b11a7983-dd5e-469b-861b-7288e5e9495a" containerName="cleanup" Jul 16 12:49:36.468995 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468503 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e45a7c5e-adfc-4bc8-a904-25163dbbd9ed" containerName="cleanup" Jul 16 12:49:36.468995 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468509 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="609b6a64-2004-4043-8c79-d34ba775b4c9" containerName="cleanup" Jul 16 12:49:36.468995 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468513 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="56f41862-a1ee-458f-9aff-e3801c36a42b" containerName="cleanup" Jul 16 12:49:36.468995 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468518 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd12cbbf-e04b-48e1-8540-cb93e5e7ff40" containerName="cleanup" Jul 16 12:49:36.468995 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.468523 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="39d45b32-de65-4feb-97d5-7a89935919bb" containerName="cleanup" Jul 16 12:49:36.471551 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.471536 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115474801200" Jul 16 12:49:36.473555 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.473539 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5mfzd\"" Jul 16 12:49:36.478870 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.478777 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-140115474801200"] Jul 16 12:49:36.599649 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.599621 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-928cl\" (UniqueName: \"kubernetes.io/projected/bf9bf30c-981e-42a2-b87a-6d5b9f2d593c-kube-api-access-928cl\") pod \"test-curl-7843-140115474801200\" (UID: \"bf9bf30c-981e-42a2-b87a-6d5b9f2d593c\") " pod="opendatahub/test-curl-7843-140115474801200" Jul 16 12:49:36.700325 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.700296 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-928cl\" (UniqueName: \"kubernetes.io/projected/bf9bf30c-981e-42a2-b87a-6d5b9f2d593c-kube-api-access-928cl\") pod \"test-curl-7843-140115474801200\" (UID: \"bf9bf30c-981e-42a2-b87a-6d5b9f2d593c\") " pod="opendatahub/test-curl-7843-140115474801200" Jul 16 12:49:36.709817 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.709795 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-928cl\" (UniqueName: \"kubernetes.io/projected/bf9bf30c-981e-42a2-b87a-6d5b9f2d593c-kube-api-access-928cl\") pod \"test-curl-7843-140115474801200\" (UID: \"bf9bf30c-981e-42a2-b87a-6d5b9f2d593c\") " pod="opendatahub/test-curl-7843-140115474801200" Jul 16 12:49:36.781996 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.781970 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115474801200" Jul 16 12:49:36.913370 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.913342 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-140115474801200"] Jul 16 12:49:36.915446 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:49:36.915394 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf9bf30c_981e_42a2_b87a_6d5b9f2d593c.slice/crio-34502506ace16029cbc7f1b52d04b2529b018c810ee0475bcd7e265378554000 WatchSource:0}: Error finding container 34502506ace16029cbc7f1b52d04b2529b018c810ee0475bcd7e265378554000: Status 404 returned error can't find the container with id 34502506ace16029cbc7f1b52d04b2529b018c810ee0475bcd7e265378554000 Jul 16 12:49:36.943531 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:36.943508 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:49:37.044302 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:37.044220 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:49:37.488619 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:37.488543 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-140115474801200" event={"ID":"bf9bf30c-981e-42a2-b87a-6d5b9f2d593c","Type":"ContainerStarted","Data":"34502506ace16029cbc7f1b52d04b2529b018c810ee0475bcd7e265378554000"} Jul 16 12:49:38.495099 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:38.495060 2578 generic.go:356] "Generic (PLEG): container finished" podID="bf9bf30c-981e-42a2-b87a-6d5b9f2d593c" containerID="2aaf4bd148b36bec8dd50047f66e6dee8b4886d6c0e1f177dc6fafbee6580180" exitCode=0 Jul 16 12:49:38.495478 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:38.495111 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-140115474801200" event={"ID":"bf9bf30c-981e-42a2-b87a-6d5b9f2d593c","Type":"ContainerDied","Data":"2aaf4bd148b36bec8dd50047f66e6dee8b4886d6c0e1f177dc6fafbee6580180"} Jul 16 12:49:39.632727 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:39.632699 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115474801200" Jul 16 12:49:39.654892 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:39.654870 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7843-140115474801200_bf9bf30c-981e-42a2-b87a-6d5b9f2d593c/test-curl-7843-140115474801200/0.log" Jul 16 12:49:39.684262 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:39.684231 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7843-140115474801200"] Jul 16 12:49:39.688036 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:39.688010 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7843-140115474801200"] Jul 16 12:49:39.731478 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:39.731455 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-928cl\" (UniqueName: \"kubernetes.io/projected/bf9bf30c-981e-42a2-b87a-6d5b9f2d593c-kube-api-access-928cl\") pod \"bf9bf30c-981e-42a2-b87a-6d5b9f2d593c\" (UID: \"bf9bf30c-981e-42a2-b87a-6d5b9f2d593c\") " Jul 16 12:49:39.733511 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:39.733490 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf9bf30c-981e-42a2-b87a-6d5b9f2d593c-kube-api-access-928cl" (OuterVolumeSpecName: "kube-api-access-928cl") pod "bf9bf30c-981e-42a2-b87a-6d5b9f2d593c" (UID: "bf9bf30c-981e-42a2-b87a-6d5b9f2d593c"). InnerVolumeSpecName "kube-api-access-928cl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:49:39.824670 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:39.824637 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7843-140115474862384"] Jul 16 12:49:39.825055 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:39.825042 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf9bf30c-981e-42a2-b87a-6d5b9f2d593c" containerName="test-curl-7843-140115474801200" Jul 16 12:49:39.825103 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:39.825057 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf9bf30c-981e-42a2-b87a-6d5b9f2d593c" containerName="test-curl-7843-140115474801200" Jul 16 12:49:39.825137 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:39.825127 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf9bf30c-981e-42a2-b87a-6d5b9f2d593c" containerName="test-curl-7843-140115474801200" Jul 16 12:49:39.828267 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:39.828249 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115474862384" Jul 16 12:49:39.832762 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:39.832733 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-928cl\" (UniqueName: \"kubernetes.io/projected/bf9bf30c-981e-42a2-b87a-6d5b9f2d593c-kube-api-access-928cl\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:49:39.834823 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:39.834802 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-140115474862384"] Jul 16 12:49:39.933760 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:39.933725 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6cdc\" (UniqueName: \"kubernetes.io/projected/5fb30fb1-019a-4d91-9efb-254e125e7b95-kube-api-access-d6cdc\") pod \"test-curl-7843-140115474862384\" (UID: \"5fb30fb1-019a-4d91-9efb-254e125e7b95\") " pod="opendatahub/test-curl-7843-140115474862384" Jul 16 12:49:40.034952 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:40.034919 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6cdc\" (UniqueName: \"kubernetes.io/projected/5fb30fb1-019a-4d91-9efb-254e125e7b95-kube-api-access-d6cdc\") pod \"test-curl-7843-140115474862384\" (UID: \"5fb30fb1-019a-4d91-9efb-254e125e7b95\") " pod="opendatahub/test-curl-7843-140115474862384" Jul 16 12:49:40.043253 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:40.043223 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6cdc\" (UniqueName: \"kubernetes.io/projected/5fb30fb1-019a-4d91-9efb-254e125e7b95-kube-api-access-d6cdc\") pod \"test-curl-7843-140115474862384\" (UID: \"5fb30fb1-019a-4d91-9efb-254e125e7b95\") " pod="opendatahub/test-curl-7843-140115474862384" Jul 16 12:49:40.138568 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:40.138482 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115474862384" Jul 16 12:49:40.266989 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:40.266963 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-140115474862384"] Jul 16 12:49:40.268292 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:49:40.268270 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5fb30fb1_019a_4d91_9efb_254e125e7b95.slice/crio-8bae8298b1d8bd0faf298278069ef6a120a1e289aa2dadd550643cda157cf916 WatchSource:0}: Error finding container 8bae8298b1d8bd0faf298278069ef6a120a1e289aa2dadd550643cda157cf916: Status 404 returned error can't find the container with id 8bae8298b1d8bd0faf298278069ef6a120a1e289aa2dadd550643cda157cf916 Jul 16 12:49:40.506909 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:40.506823 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115474801200" Jul 16 12:49:40.507073 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:40.506829 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34502506ace16029cbc7f1b52d04b2529b018c810ee0475bcd7e265378554000" Jul 16 12:49:40.507946 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:40.507919 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-140115474862384" event={"ID":"5fb30fb1-019a-4d91-9efb-254e125e7b95","Type":"ContainerStarted","Data":"8bae8298b1d8bd0faf298278069ef6a120a1e289aa2dadd550643cda157cf916"} Jul 16 12:49:40.516833 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:49:40.516807 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7843-140115474801200\" is forbidden: User \"system:node:ip-10-0-140-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-160.ec2.internal' and this object" podUID="bf9bf30c-981e-42a2-b87a-6d5b9f2d593c" pod="opendatahub/test-curl-7843-140115474801200" Jul 16 12:49:41.481477 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:49:41.481429 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7843-140115474801200\" is forbidden: User \"system:node:ip-10-0-140-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-160.ec2.internal' and this object" podUID="bf9bf30c-981e-42a2-b87a-6d5b9f2d593c" pod="opendatahub/test-curl-7843-140115474801200" Jul 16 12:49:41.483197 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:41.483167 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf9bf30c-981e-42a2-b87a-6d5b9f2d593c" path="/var/lib/kubelet/pods/bf9bf30c-981e-42a2-b87a-6d5b9f2d593c/volumes" Jul 16 12:49:41.514279 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:41.514246 2578 generic.go:356] "Generic (PLEG): container finished" podID="5fb30fb1-019a-4d91-9efb-254e125e7b95" containerID="30d7611555b344032b122a21fda0f5503a0019555d419d78d391e1026da6d5dd" exitCode=0 Jul 16 12:49:41.514426 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:41.514337 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-140115474862384" event={"ID":"5fb30fb1-019a-4d91-9efb-254e125e7b95","Type":"ContainerDied","Data":"30d7611555b344032b122a21fda0f5503a0019555d419d78d391e1026da6d5dd"} Jul 16 12:49:42.645583 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:42.645560 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115474862384" Jul 16 12:49:42.666668 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:42.666647 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7843-140115474862384_5fb30fb1-019a-4d91-9efb-254e125e7b95/test-curl-7843-140115474862384/0.log" Jul 16 12:49:42.691505 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:42.691475 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7843-140115474862384"] Jul 16 12:49:42.694306 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:42.694279 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7843-140115474862384"] Jul 16 12:49:42.761543 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:42.761505 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6cdc\" (UniqueName: \"kubernetes.io/projected/5fb30fb1-019a-4d91-9efb-254e125e7b95-kube-api-access-d6cdc\") pod \"5fb30fb1-019a-4d91-9efb-254e125e7b95\" (UID: \"5fb30fb1-019a-4d91-9efb-254e125e7b95\") " Jul 16 12:49:42.763312 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:42.763291 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fb30fb1-019a-4d91-9efb-254e125e7b95-kube-api-access-d6cdc" (OuterVolumeSpecName: "kube-api-access-d6cdc") pod "5fb30fb1-019a-4d91-9efb-254e125e7b95" (UID: "5fb30fb1-019a-4d91-9efb-254e125e7b95"). InnerVolumeSpecName "kube-api-access-d6cdc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:49:42.862282 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:42.862250 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6cdc\" (UniqueName: \"kubernetes.io/projected/5fb30fb1-019a-4d91-9efb-254e125e7b95-kube-api-access-d6cdc\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:49:42.864747 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:42.864726 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7843-140115475405680"] Jul 16 12:49:42.865094 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:42.865081 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb30fb1-019a-4d91-9efb-254e125e7b95" containerName="test-curl-7843-140115474862384" Jul 16 12:49:42.865143 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:42.865095 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb30fb1-019a-4d91-9efb-254e125e7b95" containerName="test-curl-7843-140115474862384" Jul 16 12:49:42.865178 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:42.865158 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fb30fb1-019a-4d91-9efb-254e125e7b95" containerName="test-curl-7843-140115474862384" Jul 16 12:49:42.868086 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:42.868071 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115475405680" Jul 16 12:49:42.875853 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:42.875828 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-140115475405680"] Jul 16 12:49:42.962860 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:42.962824 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drgtf\" (UniqueName: \"kubernetes.io/projected/a95e54c9-4f63-41df-bb6c-795cb1c7ec0e-kube-api-access-drgtf\") pod \"test-curl-7843-140115475405680\" (UID: \"a95e54c9-4f63-41df-bb6c-795cb1c7ec0e\") " pod="opendatahub/test-curl-7843-140115475405680" Jul 16 12:49:43.063752 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:43.063714 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drgtf\" (UniqueName: \"kubernetes.io/projected/a95e54c9-4f63-41df-bb6c-795cb1c7ec0e-kube-api-access-drgtf\") pod \"test-curl-7843-140115475405680\" (UID: \"a95e54c9-4f63-41df-bb6c-795cb1c7ec0e\") " pod="opendatahub/test-curl-7843-140115475405680" Jul 16 12:49:43.073123 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:43.073100 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drgtf\" (UniqueName: \"kubernetes.io/projected/a95e54c9-4f63-41df-bb6c-795cb1c7ec0e-kube-api-access-drgtf\") pod \"test-curl-7843-140115475405680\" (UID: \"a95e54c9-4f63-41df-bb6c-795cb1c7ec0e\") " pod="opendatahub/test-curl-7843-140115475405680" Jul 16 12:49:43.177801 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:43.177720 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115475405680" Jul 16 12:49:43.307977 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:43.307952 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-140115475405680"] Jul 16 12:49:43.309888 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:49:43.309860 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda95e54c9_4f63_41df_bb6c_795cb1c7ec0e.slice/crio-50d59c0e30015285a5750111762d0709b888a2b31076b086ddc4f07a3611eb0b WatchSource:0}: Error finding container 50d59c0e30015285a5750111762d0709b888a2b31076b086ddc4f07a3611eb0b: Status 404 returned error can't find the container with id 50d59c0e30015285a5750111762d0709b888a2b31076b086ddc4f07a3611eb0b Jul 16 12:49:43.480142 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:43.480063 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fb30fb1-019a-4d91-9efb-254e125e7b95" path="/var/lib/kubelet/pods/5fb30fb1-019a-4d91-9efb-254e125e7b95/volumes" Jul 16 12:49:43.526345 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:43.526315 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-140115475405680" event={"ID":"a95e54c9-4f63-41df-bb6c-795cb1c7ec0e","Type":"ContainerStarted","Data":"50d59c0e30015285a5750111762d0709b888a2b31076b086ddc4f07a3611eb0b"} Jul 16 12:49:43.527622 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:43.527599 2578 scope.go:117] "RemoveContainer" containerID="30d7611555b344032b122a21fda0f5503a0019555d419d78d391e1026da6d5dd" Jul 16 12:49:43.527736 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:43.527609 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115474862384" Jul 16 12:49:44.535813 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:44.535784 2578 generic.go:356] "Generic (PLEG): container finished" podID="a95e54c9-4f63-41df-bb6c-795cb1c7ec0e" containerID="58e6eb4a10779d32450f288cb32eec264f5529245de5aad126625f1a49106505" exitCode=0 Jul 16 12:49:44.536186 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:44.535872 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-140115475405680" event={"ID":"a95e54c9-4f63-41df-bb6c-795cb1c7ec0e","Type":"ContainerDied","Data":"58e6eb4a10779d32450f288cb32eec264f5529245de5aad126625f1a49106505"} Jul 16 12:49:45.668061 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:45.668036 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115475405680" Jul 16 12:49:45.690374 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:45.690344 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7843-140115475405680_a95e54c9-4f63-41df-bb6c-795cb1c7ec0e/test-curl-7843-140115475405680/0.log" Jul 16 12:49:45.717708 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:45.717687 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7843-140115475405680"] Jul 16 12:49:45.720499 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:45.720476 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7843-140115475405680"] Jul 16 12:49:45.788276 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:45.788233 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drgtf\" (UniqueName: \"kubernetes.io/projected/a95e54c9-4f63-41df-bb6c-795cb1c7ec0e-kube-api-access-drgtf\") pod \"a95e54c9-4f63-41df-bb6c-795cb1c7ec0e\" (UID: \"a95e54c9-4f63-41df-bb6c-795cb1c7ec0e\") " Jul 16 12:49:45.790251 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:45.790220 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a95e54c9-4f63-41df-bb6c-795cb1c7ec0e-kube-api-access-drgtf" (OuterVolumeSpecName: "kube-api-access-drgtf") pod "a95e54c9-4f63-41df-bb6c-795cb1c7ec0e" (UID: "a95e54c9-4f63-41df-bb6c-795cb1c7ec0e"). InnerVolumeSpecName "kube-api-access-drgtf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:49:45.853080 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:45.853056 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7843-140115475405680"] Jul 16 12:49:45.853468 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:45.853454 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a95e54c9-4f63-41df-bb6c-795cb1c7ec0e" containerName="test-curl-7843-140115475405680" Jul 16 12:49:45.853515 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:45.853470 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a95e54c9-4f63-41df-bb6c-795cb1c7ec0e" containerName="test-curl-7843-140115475405680" Jul 16 12:49:45.853546 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:45.853532 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a95e54c9-4f63-41df-bb6c-795cb1c7ec0e" containerName="test-curl-7843-140115475405680" Jul 16 12:49:45.856645 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:45.856630 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115475405680" Jul 16 12:49:45.864563 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:45.864541 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-140115475405680"] Jul 16 12:49:45.889764 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:45.889740 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drgtf\" (UniqueName: \"kubernetes.io/projected/a95e54c9-4f63-41df-bb6c-795cb1c7ec0e-kube-api-access-drgtf\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:49:45.991225 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:45.991187 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9l4n\" (UniqueName: \"kubernetes.io/projected/af1dbf46-ffae-447b-a5e2-168d16fbe819-kube-api-access-r9l4n\") pod \"test-curl-7843-140115475405680\" (UID: \"af1dbf46-ffae-447b-a5e2-168d16fbe819\") " pod="opendatahub/test-curl-7843-140115475405680" Jul 16 12:49:46.092481 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:46.092371 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9l4n\" (UniqueName: \"kubernetes.io/projected/af1dbf46-ffae-447b-a5e2-168d16fbe819-kube-api-access-r9l4n\") pod \"test-curl-7843-140115475405680\" (UID: \"af1dbf46-ffae-447b-a5e2-168d16fbe819\") " pod="opendatahub/test-curl-7843-140115475405680" Jul 16 12:49:46.100530 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:46.100498 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9l4n\" (UniqueName: \"kubernetes.io/projected/af1dbf46-ffae-447b-a5e2-168d16fbe819-kube-api-access-r9l4n\") pod \"test-curl-7843-140115475405680\" (UID: \"af1dbf46-ffae-447b-a5e2-168d16fbe819\") " pod="opendatahub/test-curl-7843-140115475405680" Jul 16 12:49:46.165759 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:46.165732 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115475405680" Jul 16 12:49:46.293775 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:46.293750 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-140115475405680"] Jul 16 12:49:46.295440 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:49:46.295396 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf1dbf46_ffae_447b_a5e2_168d16fbe819.slice/crio-8f6b9f3c3a7bbf13c189e1cd6f3a1c8970f5f675ef6215377f9ef3bbdb5e60e2 WatchSource:0}: Error finding container 8f6b9f3c3a7bbf13c189e1cd6f3a1c8970f5f675ef6215377f9ef3bbdb5e60e2: Status 404 returned error can't find the container with id 8f6b9f3c3a7bbf13c189e1cd6f3a1c8970f5f675ef6215377f9ef3bbdb5e60e2 Jul 16 12:49:46.548868 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:46.548840 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115475405680" Jul 16 12:49:46.549040 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:46.548865 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50d59c0e30015285a5750111762d0709b888a2b31076b086ddc4f07a3611eb0b" Jul 16 12:49:46.550058 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:46.550023 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-140115475405680" event={"ID":"af1dbf46-ffae-447b-a5e2-168d16fbe819","Type":"ContainerStarted","Data":"8f6b9f3c3a7bbf13c189e1cd6f3a1c8970f5f675ef6215377f9ef3bbdb5e60e2"} Jul 16 12:49:46.559779 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:46.559750 2578 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7843-140115475405680" oldPodUID="a95e54c9-4f63-41df-bb6c-795cb1c7ec0e" podUID="af1dbf46-ffae-447b-a5e2-168d16fbe819" Jul 16 12:49:47.479516 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:47.479487 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a95e54c9-4f63-41df-bb6c-795cb1c7ec0e" path="/var/lib/kubelet/pods/a95e54c9-4f63-41df-bb6c-795cb1c7ec0e/volumes" Jul 16 12:49:47.556529 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:47.556498 2578 generic.go:356] "Generic (PLEG): container finished" podID="af1dbf46-ffae-447b-a5e2-168d16fbe819" containerID="4ac68d4934ea27c72869cf470057cd32cb69b8305f1e936b9d5d14f3a2647ef0" exitCode=0 Jul 16 12:49:47.556655 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:47.556549 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-140115475405680" event={"ID":"af1dbf46-ffae-447b-a5e2-168d16fbe819","Type":"ContainerDied","Data":"4ac68d4934ea27c72869cf470057cd32cb69b8305f1e936b9d5d14f3a2647ef0"} Jul 16 12:49:48.632885 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:48.632848 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:49:48.692550 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:48.692526 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115475405680" Jul 16 12:49:48.717331 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:48.717306 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7843-140115475405680_af1dbf46-ffae-447b-a5e2-168d16fbe819/test-curl-7843-140115475405680/0.log" Jul 16 12:49:48.745178 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:48.745153 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7843-140115475405680"] Jul 16 12:49:48.747655 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:48.747635 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7843-140115475405680"] Jul 16 12:49:48.817084 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:48.817005 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9l4n\" (UniqueName: \"kubernetes.io/projected/af1dbf46-ffae-447b-a5e2-168d16fbe819-kube-api-access-r9l4n\") pod \"af1dbf46-ffae-447b-a5e2-168d16fbe819\" (UID: \"af1dbf46-ffae-447b-a5e2-168d16fbe819\") " Jul 16 12:49:48.819109 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:48.819080 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af1dbf46-ffae-447b-a5e2-168d16fbe819-kube-api-access-r9l4n" (OuterVolumeSpecName: "kube-api-access-r9l4n") pod "af1dbf46-ffae-447b-a5e2-168d16fbe819" (UID: "af1dbf46-ffae-447b-a5e2-168d16fbe819"). InnerVolumeSpecName "kube-api-access-r9l4n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:49:48.887956 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:48.887932 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7843-140115470615152"] Jul 16 12:49:48.888342 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:48.888329 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af1dbf46-ffae-447b-a5e2-168d16fbe819" containerName="test-curl-7843-140115475405680" Jul 16 12:49:48.888386 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:48.888344 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="af1dbf46-ffae-447b-a5e2-168d16fbe819" containerName="test-curl-7843-140115475405680" Jul 16 12:49:48.888441 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:48.888432 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="af1dbf46-ffae-447b-a5e2-168d16fbe819" containerName="test-curl-7843-140115475405680" Jul 16 12:49:48.891488 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:48.891473 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115470615152" Jul 16 12:49:48.897305 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:48.897262 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-140115470615152"] Jul 16 12:49:48.918633 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:48.918610 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9l4n\" (UniqueName: \"kubernetes.io/projected/af1dbf46-ffae-447b-a5e2-168d16fbe819-kube-api-access-r9l4n\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:49:49.019156 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:49.019124 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlxkx\" (UniqueName: \"kubernetes.io/projected/9d8f3e57-760b-48e7-a9d0-3e9a66137362-kube-api-access-xlxkx\") pod \"test-curl-7843-140115470615152\" (UID: \"9d8f3e57-760b-48e7-a9d0-3e9a66137362\") " pod="opendatahub/test-curl-7843-140115470615152" Jul 16 12:49:49.119871 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:49.119782 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlxkx\" (UniqueName: \"kubernetes.io/projected/9d8f3e57-760b-48e7-a9d0-3e9a66137362-kube-api-access-xlxkx\") pod \"test-curl-7843-140115470615152\" (UID: \"9d8f3e57-760b-48e7-a9d0-3e9a66137362\") " pod="opendatahub/test-curl-7843-140115470615152" Jul 16 12:49:49.130707 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:49.130683 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlxkx\" (UniqueName: \"kubernetes.io/projected/9d8f3e57-760b-48e7-a9d0-3e9a66137362-kube-api-access-xlxkx\") pod \"test-curl-7843-140115470615152\" (UID: \"9d8f3e57-760b-48e7-a9d0-3e9a66137362\") " pod="opendatahub/test-curl-7843-140115470615152" Jul 16 12:49:49.200637 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:49.200609 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115470615152" Jul 16 12:49:49.480186 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:49.480107 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af1dbf46-ffae-447b-a5e2-168d16fbe819" path="/var/lib/kubelet/pods/af1dbf46-ffae-447b-a5e2-168d16fbe819/volumes" Jul 16 12:49:49.539448 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:49.539427 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-140115470615152"] Jul 16 12:49:49.541039 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:49:49.541010 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d8f3e57_760b_48e7_a9d0_3e9a66137362.slice/crio-8a7fb152a00bbd918cc47b904f694251325ad4009d502b7a5f1823e04af77e2b WatchSource:0}: Error finding container 8a7fb152a00bbd918cc47b904f694251325ad4009d502b7a5f1823e04af77e2b: Status 404 returned error can't find the container with id 8a7fb152a00bbd918cc47b904f694251325ad4009d502b7a5f1823e04af77e2b Jul 16 12:49:49.569962 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:49.569936 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-140115470615152" event={"ID":"9d8f3e57-760b-48e7-a9d0-3e9a66137362","Type":"ContainerStarted","Data":"8a7fb152a00bbd918cc47b904f694251325ad4009d502b7a5f1823e04af77e2b"} Jul 16 12:49:49.571266 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:49.571246 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115475405680" Jul 16 12:49:49.571360 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:49.571265 2578 scope.go:117] "RemoveContainer" containerID="4ac68d4934ea27c72869cf470057cd32cb69b8305f1e936b9d5d14f3a2647ef0" Jul 16 12:49:50.579916 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:50.579880 2578 generic.go:356] "Generic (PLEG): container finished" podID="9d8f3e57-760b-48e7-a9d0-3e9a66137362" containerID="acb75a0917e0dc4e3624f49511f2e2d9837373820f5869a749f05c44366fbb45" exitCode=0 Jul 16 12:49:50.579916 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:50.579919 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-140115470615152" event={"ID":"9d8f3e57-760b-48e7-a9d0-3e9a66137362","Type":"ContainerDied","Data":"acb75a0917e0dc4e3624f49511f2e2d9837373820f5869a749f05c44366fbb45"} Jul 16 12:49:51.725936 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:51.725911 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115470615152" Jul 16 12:49:51.748642 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:51.748620 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7843-140115470615152_9d8f3e57-760b-48e7-a9d0-3e9a66137362/test-curl-7843-140115470615152/0.log" Jul 16 12:49:51.778923 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:51.778899 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7843-140115470615152"] Jul 16 12:49:51.783608 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:51.783586 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7843-140115470615152"] Jul 16 12:49:51.842921 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:51.842853 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlxkx\" (UniqueName: \"kubernetes.io/projected/9d8f3e57-760b-48e7-a9d0-3e9a66137362-kube-api-access-xlxkx\") pod \"9d8f3e57-760b-48e7-a9d0-3e9a66137362\" (UID: \"9d8f3e57-760b-48e7-a9d0-3e9a66137362\") " Jul 16 12:49:51.844778 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:51.844760 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d8f3e57-760b-48e7-a9d0-3e9a66137362-kube-api-access-xlxkx" (OuterVolumeSpecName: "kube-api-access-xlxkx") pod "9d8f3e57-760b-48e7-a9d0-3e9a66137362" (UID: "9d8f3e57-760b-48e7-a9d0-3e9a66137362"). InnerVolumeSpecName "kube-api-access-xlxkx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:49:51.919304 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:51.919274 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7843-140115470615152"] Jul 16 12:49:51.919684 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:51.919671 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d8f3e57-760b-48e7-a9d0-3e9a66137362" containerName="test-curl-7843-140115470615152" Jul 16 12:49:51.919725 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:51.919686 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d8f3e57-760b-48e7-a9d0-3e9a66137362" containerName="test-curl-7843-140115470615152" Jul 16 12:49:51.919771 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:51.919756 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d8f3e57-760b-48e7-a9d0-3e9a66137362" containerName="test-curl-7843-140115470615152" Jul 16 12:49:51.922659 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:51.922644 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115470615152" Jul 16 12:49:51.932658 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:51.932636 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-140115470615152"] Jul 16 12:49:51.943488 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:51.943468 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlxkx\" (UniqueName: \"kubernetes.io/projected/9d8f3e57-760b-48e7-a9d0-3e9a66137362-kube-api-access-xlxkx\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:49:52.044590 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:52.044556 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wckwt\" (UniqueName: \"kubernetes.io/projected/3d14cc30-e60b-4855-9589-f618a767f07f-kube-api-access-wckwt\") pod \"test-curl-7843-140115470615152\" (UID: \"3d14cc30-e60b-4855-9589-f618a767f07f\") " pod="opendatahub/test-curl-7843-140115470615152" Jul 16 12:49:52.145877 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:52.145787 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wckwt\" (UniqueName: \"kubernetes.io/projected/3d14cc30-e60b-4855-9589-f618a767f07f-kube-api-access-wckwt\") pod \"test-curl-7843-140115470615152\" (UID: \"3d14cc30-e60b-4855-9589-f618a767f07f\") " pod="opendatahub/test-curl-7843-140115470615152" Jul 16 12:49:52.155239 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:52.155213 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wckwt\" (UniqueName: \"kubernetes.io/projected/3d14cc30-e60b-4855-9589-f618a767f07f-kube-api-access-wckwt\") pod \"test-curl-7843-140115470615152\" (UID: \"3d14cc30-e60b-4855-9589-f618a767f07f\") " pod="opendatahub/test-curl-7843-140115470615152" Jul 16 12:49:52.232337 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:52.232309 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115470615152" Jul 16 12:49:52.363802 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:52.363775 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-140115470615152"] Jul 16 12:49:52.364970 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:49:52.364938 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d14cc30_e60b_4855_9589_f618a767f07f.slice/crio-b7bf6a6b5572fa9ca48fddda2a0cea164922b118fbf0e6eb18e0f0afcde9c11f WatchSource:0}: Error finding container b7bf6a6b5572fa9ca48fddda2a0cea164922b118fbf0e6eb18e0f0afcde9c11f: Status 404 returned error can't find the container with id b7bf6a6b5572fa9ca48fddda2a0cea164922b118fbf0e6eb18e0f0afcde9c11f Jul 16 12:49:52.592922 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:52.592888 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-140115470615152" event={"ID":"3d14cc30-e60b-4855-9589-f618a767f07f","Type":"ContainerStarted","Data":"b7bf6a6b5572fa9ca48fddda2a0cea164922b118fbf0e6eb18e0f0afcde9c11f"} Jul 16 12:49:52.594116 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:52.594096 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115470615152" Jul 16 12:49:52.594116 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:52.594111 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a7fb152a00bbd918cc47b904f694251325ad4009d502b7a5f1823e04af77e2b" Jul 16 12:49:52.605605 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:52.605581 2578 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7843-140115470615152" oldPodUID="9d8f3e57-760b-48e7-a9d0-3e9a66137362" podUID="3d14cc30-e60b-4855-9589-f618a767f07f" Jul 16 12:49:53.479882 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:53.479849 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d8f3e57-760b-48e7-a9d0-3e9a66137362" path="/var/lib/kubelet/pods/9d8f3e57-760b-48e7-a9d0-3e9a66137362/volumes" Jul 16 12:49:53.601427 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:53.601383 2578 generic.go:356] "Generic (PLEG): container finished" podID="3d14cc30-e60b-4855-9589-f618a767f07f" containerID="f8141d27f521097d6107b493e511fb52ec86178f8b53cc6ca01524e85cba206d" exitCode=0 Jul 16 12:49:53.601591 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:53.601470 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-140115470615152" event={"ID":"3d14cc30-e60b-4855-9589-f618a767f07f","Type":"ContainerDied","Data":"f8141d27f521097d6107b493e511fb52ec86178f8b53cc6ca01524e85cba206d"} Jul 16 12:49:54.739016 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:54.738992 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115470615152" Jul 16 12:49:54.774255 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:54.774232 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7843-140115470615152_3d14cc30-e60b-4855-9589-f618a767f07f/test-curl-7843-140115470615152/0.log" Jul 16 12:49:54.802228 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:54.802198 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7843-140115470615152"] Jul 16 12:49:54.804758 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:54.804737 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7843-140115470615152"] Jul 16 12:49:54.870851 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:54.870773 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wckwt\" (UniqueName: \"kubernetes.io/projected/3d14cc30-e60b-4855-9589-f618a767f07f-kube-api-access-wckwt\") pod \"3d14cc30-e60b-4855-9589-f618a767f07f\" (UID: \"3d14cc30-e60b-4855-9589-f618a767f07f\") " Jul 16 12:49:54.872793 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:54.872765 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d14cc30-e60b-4855-9589-f618a767f07f-kube-api-access-wckwt" (OuterVolumeSpecName: "kube-api-access-wckwt") pod "3d14cc30-e60b-4855-9589-f618a767f07f" (UID: "3d14cc30-e60b-4855-9589-f618a767f07f"). InnerVolumeSpecName "kube-api-access-wckwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:49:54.971995 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:54.971970 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wckwt\" (UniqueName: \"kubernetes.io/projected/3d14cc30-e60b-4855-9589-f618a767f07f-kube-api-access-wckwt\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:49:55.479393 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:55.479351 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d14cc30-e60b-4855-9589-f618a767f07f" path="/var/lib/kubelet/pods/3d14cc30-e60b-4855-9589-f618a767f07f/volumes" Jul 16 12:49:55.491961 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:55.491940 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:49:55.614368 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:55.614340 2578 scope.go:117] "RemoveContainer" containerID="f8141d27f521097d6107b493e511fb52ec86178f8b53cc6ca01524e85cba206d" Jul 16 12:49:55.614368 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:49:55.614358 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-140115470615152" Jul 16 12:50:01.941513 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:01.941451 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:50:02.045986 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:02.045957 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:50:02.235737 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:02.235658 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx"] Jul 16 12:50:02.235979 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:02.235952 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" podUID="9d53a7c2-97e8-4278-b1db-93c0e592e186" containerName="istio-proxy" containerID="cri-o://86f63d93a38bbcaa5e3076663cfa82a867701044653f33d7c265d8f0c30310d7" gracePeriod=30 Jul 16 12:50:02.258349 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:02.258322 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:50:02.498342 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:02.498259 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:50:03.017521 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:03.017489 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:50:06.054685 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:06.054648 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:50:07.494290 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.494267 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:50:07.568812 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.568782 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d53a7c2-97e8-4278-b1db-93c0e592e186-istiod-ca-cert\") pod \"9d53a7c2-97e8-4278-b1db-93c0e592e186\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " Jul 16 12:50:07.568812 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.568818 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-envoy\") pod \"9d53a7c2-97e8-4278-b1db-93c0e592e186\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " Jul 16 12:50:07.569040 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.568852 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-data\") pod \"9d53a7c2-97e8-4278-b1db-93c0e592e186\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " Jul 16 12:50:07.569040 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.568872 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-workload-socket\") pod \"9d53a7c2-97e8-4278-b1db-93c0e592e186\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " Jul 16 12:50:07.569040 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.568940 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-credential-socket\") pod \"9d53a7c2-97e8-4278-b1db-93c0e592e186\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " Jul 16 12:50:07.569040 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.568963 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-podinfo\") pod \"9d53a7c2-97e8-4278-b1db-93c0e592e186\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " Jul 16 12:50:07.569040 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.568996 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85pd2\" (UniqueName: \"kubernetes.io/projected/9d53a7c2-97e8-4278-b1db-93c0e592e186-kube-api-access-85pd2\") pod \"9d53a7c2-97e8-4278-b1db-93c0e592e186\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " Jul 16 12:50:07.569040 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.569026 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-token\") pod \"9d53a7c2-97e8-4278-b1db-93c0e592e186\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " Jul 16 12:50:07.569352 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.569064 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-workload-certs\") pod \"9d53a7c2-97e8-4278-b1db-93c0e592e186\" (UID: \"9d53a7c2-97e8-4278-b1db-93c0e592e186\") " Jul 16 12:50:07.569352 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.569168 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9d53a7c2-97e8-4278-b1db-93c0e592e186" (UID: "9d53a7c2-97e8-4278-b1db-93c0e592e186"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:50:07.569352 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.569285 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9d53a7c2-97e8-4278-b1db-93c0e592e186" (UID: "9d53a7c2-97e8-4278-b1db-93c0e592e186"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:50:07.569352 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.569288 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d53a7c2-97e8-4278-b1db-93c0e592e186-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9d53a7c2-97e8-4278-b1db-93c0e592e186" (UID: "9d53a7c2-97e8-4278-b1db-93c0e592e186"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:50:07.569597 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.569382 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9d53a7c2-97e8-4278-b1db-93c0e592e186" (UID: "9d53a7c2-97e8-4278-b1db-93c0e592e186"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:50:07.569597 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.569465 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d53a7c2-97e8-4278-b1db-93c0e592e186-istiod-ca-cert\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:07.569597 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.569486 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-workload-socket\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:07.569597 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.569501 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-credential-socket\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:07.569818 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.569790 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-data" (OuterVolumeSpecName: "istio-data") pod "9d53a7c2-97e8-4278-b1db-93c0e592e186" (UID: "9d53a7c2-97e8-4278-b1db-93c0e592e186"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:50:07.571007 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.570989 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9d53a7c2-97e8-4278-b1db-93c0e592e186" (UID: "9d53a7c2-97e8-4278-b1db-93c0e592e186"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:50:07.571283 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.571269 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-token" (OuterVolumeSpecName: "istio-token") pod "9d53a7c2-97e8-4278-b1db-93c0e592e186" (UID: "9d53a7c2-97e8-4278-b1db-93c0e592e186"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:50:07.571391 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.571378 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9d53a7c2-97e8-4278-b1db-93c0e592e186" (UID: "9d53a7c2-97e8-4278-b1db-93c0e592e186"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 12:50:07.571544 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.571522 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d53a7c2-97e8-4278-b1db-93c0e592e186-kube-api-access-85pd2" (OuterVolumeSpecName: "kube-api-access-85pd2") pod "9d53a7c2-97e8-4278-b1db-93c0e592e186" (UID: "9d53a7c2-97e8-4278-b1db-93c0e592e186"). InnerVolumeSpecName "kube-api-access-85pd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:50:07.670409 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.670381 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-podinfo\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:07.670409 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.670407 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85pd2\" (UniqueName: \"kubernetes.io/projected/9d53a7c2-97e8-4278-b1db-93c0e592e186-kube-api-access-85pd2\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:07.670601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.670445 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-token\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:07.670601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.670458 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-workload-certs\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:07.670601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.670469 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-envoy\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:07.670601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.670479 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d53a7c2-97e8-4278-b1db-93c0e592e186-istio-data\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:07.686054 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.686025 2578 generic.go:356] "Generic (PLEG): container finished" podID="9d53a7c2-97e8-4278-b1db-93c0e592e186" containerID="86f63d93a38bbcaa5e3076663cfa82a867701044653f33d7c265d8f0c30310d7" exitCode=0 Jul 16 12:50:07.686176 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.686069 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" Jul 16 12:50:07.686176 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.686074 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" event={"ID":"9d53a7c2-97e8-4278-b1db-93c0e592e186","Type":"ContainerDied","Data":"86f63d93a38bbcaa5e3076663cfa82a867701044653f33d7c265d8f0c30310d7"} Jul 16 12:50:07.686176 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.686125 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx" event={"ID":"9d53a7c2-97e8-4278-b1db-93c0e592e186","Type":"ContainerDied","Data":"789e11523407bcfbcbde67f84fd3207399186356ce04f85fb28ac5638b71c6a9"} Jul 16 12:50:07.686176 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.686148 2578 scope.go:117] "RemoveContainer" containerID="86f63d93a38bbcaa5e3076663cfa82a867701044653f33d7c265d8f0c30310d7" Jul 16 12:50:07.696044 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.696028 2578 scope.go:117] "RemoveContainer" containerID="64da36fb39c24e0f7974419953a52000d86c69d227fd85b5f5a7157cec485678" Jul 16 12:50:07.704129 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.704112 2578 scope.go:117] "RemoveContainer" containerID="86f63d93a38bbcaa5e3076663cfa82a867701044653f33d7c265d8f0c30310d7" Jul 16 12:50:07.704347 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:50:07.704330 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86f63d93a38bbcaa5e3076663cfa82a867701044653f33d7c265d8f0c30310d7\": container with ID starting with 86f63d93a38bbcaa5e3076663cfa82a867701044653f33d7c265d8f0c30310d7 not found: ID does not exist" containerID="86f63d93a38bbcaa5e3076663cfa82a867701044653f33d7c265d8f0c30310d7" Jul 16 12:50:07.704428 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.704351 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86f63d93a38bbcaa5e3076663cfa82a867701044653f33d7c265d8f0c30310d7"} err="failed to get container status \"86f63d93a38bbcaa5e3076663cfa82a867701044653f33d7c265d8f0c30310d7\": rpc error: code = NotFound desc = could not find container \"86f63d93a38bbcaa5e3076663cfa82a867701044653f33d7c265d8f0c30310d7\": container with ID starting with 86f63d93a38bbcaa5e3076663cfa82a867701044653f33d7c265d8f0c30310d7 not found: ID does not exist" Jul 16 12:50:07.704428 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.704367 2578 scope.go:117] "RemoveContainer" containerID="64da36fb39c24e0f7974419953a52000d86c69d227fd85b5f5a7157cec485678" Jul 16 12:50:07.704625 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:50:07.704608 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64da36fb39c24e0f7974419953a52000d86c69d227fd85b5f5a7157cec485678\": container with ID starting with 64da36fb39c24e0f7974419953a52000d86c69d227fd85b5f5a7157cec485678 not found: ID does not exist" containerID="64da36fb39c24e0f7974419953a52000d86c69d227fd85b5f5a7157cec485678" Jul 16 12:50:07.704671 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.704631 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64da36fb39c24e0f7974419953a52000d86c69d227fd85b5f5a7157cec485678"} err="failed to get container status \"64da36fb39c24e0f7974419953a52000d86c69d227fd85b5f5a7157cec485678\": rpc error: code = NotFound desc = could not find container \"64da36fb39c24e0f7974419953a52000d86c69d227fd85b5f5a7157cec485678\": container with ID starting with 64da36fb39c24e0f7974419953a52000d86c69d227fd85b5f5a7157cec485678 not found: ID does not exist" Jul 16 12:50:07.708905 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.708886 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx"] Jul 16 12:50:07.715903 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:07.715874 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-cf65e4-openshift-default-5cd777b55f-92qhx"] Jul 16 12:50:09.112563 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:09.112534 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:50:09.213524 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:09.213492 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:50:09.376306 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:09.376221 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8"] Jul 16 12:50:09.376540 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:09.376514 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" podUID="6a498dfe-73c9-45dd-8456-b9012fc0254f" containerName="istio-proxy" containerID="cri-o://b96a080da4a8904eb0e01415615b5fd403429769524c3e734d784d5ea492b062" gracePeriod=30 Jul 16 12:50:09.402101 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:09.402073 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:50:09.480670 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:09.480639 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d53a7c2-97e8-4278-b1db-93c0e592e186" path="/var/lib/kubelet/pods/9d53a7c2-97e8-4278-b1db-93c0e592e186/volumes" Jul 16 12:50:09.656533 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:09.656406 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:50:13.630744 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:13.630710 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:50:14.643818 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.643758 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:50:14.729301 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.729272 2578 generic.go:356] "Generic (PLEG): container finished" podID="6a498dfe-73c9-45dd-8456-b9012fc0254f" containerID="b96a080da4a8904eb0e01415615b5fd403429769524c3e734d784d5ea492b062" exitCode=0 Jul 16 12:50:14.729449 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.729321 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" Jul 16 12:50:14.729449 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.729361 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" event={"ID":"6a498dfe-73c9-45dd-8456-b9012fc0254f","Type":"ContainerDied","Data":"b96a080da4a8904eb0e01415615b5fd403429769524c3e734d784d5ea492b062"} Jul 16 12:50:14.729449 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.729430 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8" event={"ID":"6a498dfe-73c9-45dd-8456-b9012fc0254f","Type":"ContainerDied","Data":"73be57e58784a07841bd7e581e9ef798a84c2db8999d0751bb81e3032e924891"} Jul 16 12:50:14.729580 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.729456 2578 scope.go:117] "RemoveContainer" containerID="b96a080da4a8904eb0e01415615b5fd403429769524c3e734d784d5ea492b062" Jul 16 12:50:14.732803 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.732781 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-token\") pod \"6a498dfe-73c9-45dd-8456-b9012fc0254f\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " Jul 16 12:50:14.732934 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.732826 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-podinfo\") pod \"6a498dfe-73c9-45dd-8456-b9012fc0254f\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " Jul 16 12:50:14.732934 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.732896 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-data\") pod \"6a498dfe-73c9-45dd-8456-b9012fc0254f\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " Jul 16 12:50:14.732934 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.732919 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-envoy\") pod \"6a498dfe-73c9-45dd-8456-b9012fc0254f\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " Jul 16 12:50:14.733091 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.732947 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-workload-socket\") pod \"6a498dfe-73c9-45dd-8456-b9012fc0254f\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " Jul 16 12:50:14.733091 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.732978 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a498dfe-73c9-45dd-8456-b9012fc0254f-istiod-ca-cert\") pod \"6a498dfe-73c9-45dd-8456-b9012fc0254f\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " Jul 16 12:50:14.733091 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.733005 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-workload-certs\") pod \"6a498dfe-73c9-45dd-8456-b9012fc0254f\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " Jul 16 12:50:14.733091 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.733042 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzdqn\" (UniqueName: \"kubernetes.io/projected/6a498dfe-73c9-45dd-8456-b9012fc0254f-kube-api-access-zzdqn\") pod \"6a498dfe-73c9-45dd-8456-b9012fc0254f\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " Jul 16 12:50:14.733091 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.733082 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-credential-socket\") pod \"6a498dfe-73c9-45dd-8456-b9012fc0254f\" (UID: \"6a498dfe-73c9-45dd-8456-b9012fc0254f\") " Jul 16 12:50:14.733397 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.733310 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6a498dfe-73c9-45dd-8456-b9012fc0254f" (UID: "6a498dfe-73c9-45dd-8456-b9012fc0254f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:50:14.733499 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.733386 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a498dfe-73c9-45dd-8456-b9012fc0254f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6a498dfe-73c9-45dd-8456-b9012fc0254f" (UID: "6a498dfe-73c9-45dd-8456-b9012fc0254f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:50:14.733609 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.733586 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6a498dfe-73c9-45dd-8456-b9012fc0254f" (UID: "6a498dfe-73c9-45dd-8456-b9012fc0254f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:50:14.733721 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.733597 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6a498dfe-73c9-45dd-8456-b9012fc0254f" (UID: "6a498dfe-73c9-45dd-8456-b9012fc0254f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:50:14.733797 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.733770 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-data" (OuterVolumeSpecName: "istio-data") pod "6a498dfe-73c9-45dd-8456-b9012fc0254f" (UID: "6a498dfe-73c9-45dd-8456-b9012fc0254f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:50:14.735211 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.735178 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6a498dfe-73c9-45dd-8456-b9012fc0254f" (UID: "6a498dfe-73c9-45dd-8456-b9012fc0254f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 12:50:14.735363 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.735346 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-token" (OuterVolumeSpecName: "istio-token") pod "6a498dfe-73c9-45dd-8456-b9012fc0254f" (UID: "6a498dfe-73c9-45dd-8456-b9012fc0254f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:50:14.735455 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.735408 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6a498dfe-73c9-45dd-8456-b9012fc0254f" (UID: "6a498dfe-73c9-45dd-8456-b9012fc0254f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:50:14.735643 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.735613 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a498dfe-73c9-45dd-8456-b9012fc0254f-kube-api-access-zzdqn" (OuterVolumeSpecName: "kube-api-access-zzdqn") pod "6a498dfe-73c9-45dd-8456-b9012fc0254f" (UID: "6a498dfe-73c9-45dd-8456-b9012fc0254f"). InnerVolumeSpecName "kube-api-access-zzdqn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:50:14.739317 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.739291 2578 scope.go:117] "RemoveContainer" containerID="57aa2c39f8845d8ef1490ab7a2aea02313f967dfbc705e1f24e10c30d107358b" Jul 16 12:50:14.753207 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.753188 2578 scope.go:117] "RemoveContainer" containerID="b96a080da4a8904eb0e01415615b5fd403429769524c3e734d784d5ea492b062" Jul 16 12:50:14.753477 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:50:14.753456 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b96a080da4a8904eb0e01415615b5fd403429769524c3e734d784d5ea492b062\": container with ID starting with b96a080da4a8904eb0e01415615b5fd403429769524c3e734d784d5ea492b062 not found: ID does not exist" containerID="b96a080da4a8904eb0e01415615b5fd403429769524c3e734d784d5ea492b062" Jul 16 12:50:14.753535 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.753488 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b96a080da4a8904eb0e01415615b5fd403429769524c3e734d784d5ea492b062"} err="failed to get container status \"b96a080da4a8904eb0e01415615b5fd403429769524c3e734d784d5ea492b062\": rpc error: code = NotFound desc = could not find container \"b96a080da4a8904eb0e01415615b5fd403429769524c3e734d784d5ea492b062\": container with ID starting with b96a080da4a8904eb0e01415615b5fd403429769524c3e734d784d5ea492b062 not found: ID does not exist" Jul 16 12:50:14.753535 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.753508 2578 scope.go:117] "RemoveContainer" containerID="57aa2c39f8845d8ef1490ab7a2aea02313f967dfbc705e1f24e10c30d107358b" Jul 16 12:50:14.753770 ip-10-0-140-160 kubenswrapper[2578]: E0716 12:50:14.753748 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57aa2c39f8845d8ef1490ab7a2aea02313f967dfbc705e1f24e10c30d107358b\": container with ID starting with 57aa2c39f8845d8ef1490ab7a2aea02313f967dfbc705e1f24e10c30d107358b not found: ID does not exist" containerID="57aa2c39f8845d8ef1490ab7a2aea02313f967dfbc705e1f24e10c30d107358b" Jul 16 12:50:14.753851 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.753773 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57aa2c39f8845d8ef1490ab7a2aea02313f967dfbc705e1f24e10c30d107358b"} err="failed to get container status \"57aa2c39f8845d8ef1490ab7a2aea02313f967dfbc705e1f24e10c30d107358b\": rpc error: code = NotFound desc = could not find container \"57aa2c39f8845d8ef1490ab7a2aea02313f967dfbc705e1f24e10c30d107358b\": container with ID starting with 57aa2c39f8845d8ef1490ab7a2aea02313f967dfbc705e1f24e10c30d107358b not found: ID does not exist" Jul 16 12:50:14.834594 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.834565 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-data\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:14.834594 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.834589 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-envoy\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:14.834756 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.834601 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-workload-socket\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:14.834756 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.834614 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a498dfe-73c9-45dd-8456-b9012fc0254f-istiod-ca-cert\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:14.834756 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.834625 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-workload-certs\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:14.834756 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.834636 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzdqn\" (UniqueName: \"kubernetes.io/projected/6a498dfe-73c9-45dd-8456-b9012fc0254f-kube-api-access-zzdqn\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:14.834756 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.834648 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a498dfe-73c9-45dd-8456-b9012fc0254f-credential-socket\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:14.834756 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.834659 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-token\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:14.834756 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:14.834670 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a498dfe-73c9-45dd-8456-b9012fc0254f-istio-podinfo\") on node \"ip-10-0-140-160.ec2.internal\" DevicePath \"\"" Jul 16 12:50:15.057728 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:15.057699 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8"] Jul 16 12:50:15.062966 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:15.062944 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-3c72a3-openshift-default-ff964cf99-4t4s8"] Jul 16 12:50:15.480441 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:15.480345 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a498dfe-73c9-45dd-8456-b9012fc0254f" path="/var/lib/kubelet/pods/6a498dfe-73c9-45dd-8456-b9012fc0254f/volumes" Jul 16 12:50:17.079225 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:17.079192 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm_92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0/copy-manifests/0.log" Jul 16 12:50:17.101074 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:17.101045 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm_92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0/manager/0.log" Jul 16 12:50:17.645013 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:17.644968 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-e2e-ait-c7727fcf-684d45b745-c6lsd_be2bf342-ab6e-498f-b302-aeeb783fe9d4/maas-api/0.log" Jul 16 12:50:19.141625 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:19.141594 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cs4r8"] Jul 16 12:50:19.171013 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:19.170990 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-ait-c7727fcf-29736765-j59kd_39d45b32-de65-4feb-97d5-7a89935919bb/cleanup/0.log" Jul 16 12:50:19.300338 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:19.300310 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-api-a-70ba56-29736765-7vssx_e45a7c5e-adfc-4bc8-a904-25163dbbd9ed/cleanup/0.log" Jul 16 12:50:19.426140 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:19.426067 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-api-b-47e820-29736765-krvb2_b11a7983-dd5e-469b-861b-7288e5e9495a/cleanup/0.log" Jul 16 12:50:19.556266 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:19.556241 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-derive-15231c4d-29736765-7gc2f_bd12cbbf-e04b-48e1-8540-cb93e5e7ff40/cleanup/0.log" Jul 16 12:50:19.676547 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:19.676474 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-inf-a-28d017-29736765-8krjw_97365f2c-f17c-4b52-adb5-6bdcf641dd6b/cleanup/0.log" Jul 16 12:50:19.912576 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:19.912547 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-migrate-051a7b7b-29736765-p5plb_56f41862-a1ee-458f-9aff-e3801c36a42b/cleanup/0.log" Jul 16 12:50:20.033075 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:20.033051 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-mt-df8d3cf1-29736765-wpfs2_609b6a64-2004-4043-8c79-d34ba775b4c9/cleanup/0.log" Jul 16 12:50:20.154645 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:20.154616 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-shared-a-cf65e4-29736765-g74dp_71ec1322-aa86-4615-af84-16b7ddd82014/cleanup/0.log" Jul 16 12:50:20.276176 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:20.276151 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-shared-b-3c72a3-29736765-p9vjf_0cd43e61-cc0a-4c2d-beef-06905c7b2486/cleanup/0.log" Jul 16 12:50:21.277226 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:21.277197 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k_e13484fb-1430-473c-9b88-0d0fbd8e76cc/util/0.log" Jul 16 12:50:21.284675 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:21.284656 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k_e13484fb-1430-473c-9b88-0d0fbd8e76cc/pull/0.log" Jul 16 12:50:21.291925 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:21.291904 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k_e13484fb-1430-473c-9b88-0d0fbd8e76cc/extract/0.log" Jul 16 12:50:21.423455 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:21.423430 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq_c09bf55c-8a63-49eb-9f5a-d09ce780a38a/util/0.log" Jul 16 12:50:21.431357 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:21.431334 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq_c09bf55c-8a63-49eb-9f5a-d09ce780a38a/pull/0.log" Jul 16 12:50:21.439590 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:21.439561 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq_c09bf55c-8a63-49eb-9f5a-d09ce780a38a/extract/0.log" Jul 16 12:50:21.566775 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:21.566752 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k_1323e7ab-7964-460c-a2d0-77fbced802b5/util/0.log" Jul 16 12:50:21.574655 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:21.574634 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k_1323e7ab-7964-460c-a2d0-77fbced802b5/pull/0.log" Jul 16 12:50:21.585164 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:21.585140 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k_1323e7ab-7964-460c-a2d0-77fbced802b5/extract/0.log" Jul 16 12:50:21.713118 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:21.713091 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s_f5bf6abd-db25-4f50-b5a2-49def53cb4af/util/0.log" Jul 16 12:50:21.720531 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:21.720511 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s_f5bf6abd-db25-4f50-b5a2-49def53cb4af/pull/0.log" Jul 16 12:50:21.727672 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:21.727653 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s_f5bf6abd-db25-4f50-b5a2-49def53cb4af/extract/0.log" Jul 16 12:50:22.660380 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:22.660353 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-cs4r8_f1e16a12-c1d6-49c2-bc1c-325dccb707bb/limitador/0.log" Jul 16 12:50:23.355719 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:23.355682 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6lsdz_956ffded-32d6-416b-baca-9de1b5cc5747/discovery/0.log" Jul 16 12:50:24.108744 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:24.108713 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68956567b4-ccm7k_060e6722-ce48-468c-aa89-cc07918db901/router/0.log" Jul 16 12:50:31.497952 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.497917 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xm7nj/must-gather-7kz6d"] Jul 16 12:50:31.498324 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.498280 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d53a7c2-97e8-4278-b1db-93c0e592e186" containerName="istio-proxy" Jul 16 12:50:31.498324 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.498290 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d53a7c2-97e8-4278-b1db-93c0e592e186" containerName="istio-proxy" Jul 16 12:50:31.498324 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.498299 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a498dfe-73c9-45dd-8456-b9012fc0254f" containerName="istio-proxy" Jul 16 12:50:31.498324 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.498304 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a498dfe-73c9-45dd-8456-b9012fc0254f" containerName="istio-proxy" Jul 16 12:50:31.498324 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.498317 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a498dfe-73c9-45dd-8456-b9012fc0254f" containerName="istio-proxy" Jul 16 12:50:31.498324 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.498321 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a498dfe-73c9-45dd-8456-b9012fc0254f" containerName="istio-proxy" Jul 16 12:50:31.498548 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.498335 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d14cc30-e60b-4855-9589-f618a767f07f" containerName="test-curl-7843-140115470615152" Jul 16 12:50:31.498548 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.498342 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d14cc30-e60b-4855-9589-f618a767f07f" containerName="test-curl-7843-140115470615152" Jul 16 12:50:31.498548 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.498397 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d53a7c2-97e8-4278-b1db-93c0e592e186" containerName="istio-proxy" Jul 16 12:50:31.498548 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.498404 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a498dfe-73c9-45dd-8456-b9012fc0254f" containerName="istio-proxy" Jul 16 12:50:31.498548 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.498434 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a498dfe-73c9-45dd-8456-b9012fc0254f" containerName="istio-proxy" Jul 16 12:50:31.498548 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.498442 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d14cc30-e60b-4855-9589-f618a767f07f" containerName="test-curl-7843-140115470615152" Jul 16 12:50:31.498548 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.498499 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d53a7c2-97e8-4278-b1db-93c0e592e186" containerName="istio-proxy" Jul 16 12:50:31.498548 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.498504 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d53a7c2-97e8-4278-b1db-93c0e592e186" containerName="istio-proxy" Jul 16 12:50:31.498799 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.498560 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d53a7c2-97e8-4278-b1db-93c0e592e186" containerName="istio-proxy" Jul 16 12:50:31.503246 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.503222 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xm7nj/must-gather-7kz6d" Jul 16 12:50:31.508514 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.508492 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xm7nj\"/\"default-dockercfg-7f7h2\"" Jul 16 12:50:31.508644 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.508500 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xm7nj\"/\"openshift-service-ca.crt\"" Jul 16 12:50:31.509034 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.509009 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xm7nj\"/\"kube-root-ca.crt\"" Jul 16 12:50:31.513144 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.513119 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xm7nj/must-gather-7kz6d"] Jul 16 12:50:31.513714 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.513694 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:50:31.578025 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.577989 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dc710d0a-b59b-4801-a205-82a1927884a5-must-gather-output\") pod \"must-gather-7kz6d\" (UID: \"dc710d0a-b59b-4801-a205-82a1927884a5\") " pod="openshift-must-gather-xm7nj/must-gather-7kz6d" Jul 16 12:50:31.578025 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.578028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krmwm\" (UniqueName: \"kubernetes.io/projected/dc710d0a-b59b-4801-a205-82a1927884a5-kube-api-access-krmwm\") pod \"must-gather-7kz6d\" (UID: \"dc710d0a-b59b-4801-a205-82a1927884a5\") " pod="openshift-must-gather-xm7nj/must-gather-7kz6d" Jul 16 12:50:31.606243 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.606203 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" limit=1024 Jul 16 12:50:31.678748 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.678715 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dc710d0a-b59b-4801-a205-82a1927884a5-must-gather-output\") pod \"must-gather-7kz6d\" (UID: \"dc710d0a-b59b-4801-a205-82a1927884a5\") " pod="openshift-must-gather-xm7nj/must-gather-7kz6d" Jul 16 12:50:31.678748 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.678746 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krmwm\" (UniqueName: \"kubernetes.io/projected/dc710d0a-b59b-4801-a205-82a1927884a5-kube-api-access-krmwm\") pod \"must-gather-7kz6d\" (UID: \"dc710d0a-b59b-4801-a205-82a1927884a5\") " pod="openshift-must-gather-xm7nj/must-gather-7kz6d" Jul 16 12:50:31.679044 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.679023 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dc710d0a-b59b-4801-a205-82a1927884a5-must-gather-output\") pod \"must-gather-7kz6d\" (UID: \"dc710d0a-b59b-4801-a205-82a1927884a5\") " pod="openshift-must-gather-xm7nj/must-gather-7kz6d" Jul 16 12:50:31.687005 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.686983 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krmwm\" (UniqueName: \"kubernetes.io/projected/dc710d0a-b59b-4801-a205-82a1927884a5-kube-api-access-krmwm\") pod \"must-gather-7kz6d\" (UID: \"dc710d0a-b59b-4801-a205-82a1927884a5\") " pod="openshift-must-gather-xm7nj/must-gather-7kz6d" Jul 16 12:50:31.813252 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.813216 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xm7nj/must-gather-7kz6d" Jul 16 12:50:31.947391 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.947367 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xm7nj/must-gather-7kz6d"] Jul 16 12:50:31.949200 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:50:31.949173 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc710d0a_b59b_4801_a205_82a1927884a5.slice/crio-86ed28bd0193d3378ab86357ce3e7a3b534ea1c7617bdf906b95a517cd078fb5 WatchSource:0}: Error finding container 86ed28bd0193d3378ab86357ce3e7a3b534ea1c7617bdf906b95a517cd078fb5: Status 404 returned error can't find the container with id 86ed28bd0193d3378ab86357ce3e7a3b534ea1c7617bdf906b95a517cd078fb5 Jul 16 12:50:31.950963 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:31.950946 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 12:50:32.836129 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:32.836099 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xm7nj/must-gather-7kz6d" event={"ID":"dc710d0a-b59b-4801-a205-82a1927884a5","Type":"ContainerStarted","Data":"86ed28bd0193d3378ab86357ce3e7a3b534ea1c7617bdf906b95a517cd078fb5"} Jul 16 12:50:33.847931 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:33.847897 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xm7nj/must-gather-7kz6d" event={"ID":"dc710d0a-b59b-4801-a205-82a1927884a5","Type":"ContainerStarted","Data":"518720fa476a5b963e4367e0946e9685b209afbd07e274f93db052f4d824b1dc"} Jul 16 12:50:33.847931 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:33.847936 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xm7nj/must-gather-7kz6d" event={"ID":"dc710d0a-b59b-4801-a205-82a1927884a5","Type":"ContainerStarted","Data":"be15cc90f853071fecd026ae25f4f762813da9b30240c786063a474e75896cfe"} Jul 16 12:50:33.869867 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:33.869814 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xm7nj/must-gather-7kz6d" podStartSLOduration=1.9260579629999999 podStartE2EDuration="2.869797697s" totalImagesPullingTime="943.739734ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:50:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:50:33.866557968 +0000 UTC m=+2583.629927427" watchObservedRunningTime="2026-07-16 12:50:33.869797697 +0000 UTC m=+2583.633167199" Jul 16 12:50:34.645533 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:34.645503 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mq859_a0853c65-0aa1-4d5f-aab4-345f5d30f85f/global-pull-secret-syncer/0.log" Jul 16 12:50:34.703091 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:34.703065 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9p5kr_6fc5ae7e-11cb-4716-8341-d6613923b14b/konnectivity-agent/0.log" Jul 16 12:50:34.821912 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:34.821886 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-160.ec2.internal_4e3b2dbb5e31a37252e72dba16a64d8f/haproxy/0.log" Jul 16 12:50:39.126600 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:39.126560 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k_e13484fb-1430-473c-9b88-0d0fbd8e76cc/extract/0.log" Jul 16 12:50:39.166872 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:39.166844 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k_e13484fb-1430-473c-9b88-0d0fbd8e76cc/util/0.log" Jul 16 12:50:39.192999 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:39.192974 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n4s6k_e13484fb-1430-473c-9b88-0d0fbd8e76cc/pull/0.log" Jul 16 12:50:39.224501 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:39.224441 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq_c09bf55c-8a63-49eb-9f5a-d09ce780a38a/extract/0.log" Jul 16 12:50:39.252729 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:39.252705 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq_c09bf55c-8a63-49eb-9f5a-d09ce780a38a/util/0.log" Jul 16 12:50:39.278505 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:39.278473 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mw8xq_c09bf55c-8a63-49eb-9f5a-d09ce780a38a/pull/0.log" Jul 16 12:50:39.312085 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:39.312042 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k_1323e7ab-7964-460c-a2d0-77fbced802b5/extract/0.log" Jul 16 12:50:39.342263 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:39.342235 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k_1323e7ab-7964-460c-a2d0-77fbced802b5/util/0.log" Jul 16 12:50:39.373116 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:39.373090 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h8d8k_1323e7ab-7964-460c-a2d0-77fbced802b5/pull/0.log" Jul 16 12:50:39.414134 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:39.406468 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s_f5bf6abd-db25-4f50-b5a2-49def53cb4af/extract/0.log" Jul 16 12:50:39.447175 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:39.447149 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s_f5bf6abd-db25-4f50-b5a2-49def53cb4af/util/0.log" Jul 16 12:50:39.478233 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:39.476854 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wml7s_f5bf6abd-db25-4f50-b5a2-49def53cb4af/pull/0.log" Jul 16 12:50:40.156154 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:40.156124 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-cs4r8_f1e16a12-c1d6-49c2-bc1c-325dccb707bb/limitador/0.log" Jul 16 12:50:41.860145 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:41.860119 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_491953d0-3e2f-4b08-aacf-316d9631c006/alertmanager/0.log" Jul 16 12:50:41.896188 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:41.896153 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_491953d0-3e2f-4b08-aacf-316d9631c006/config-reloader/0.log" Jul 16 12:50:41.922539 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:41.922492 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_491953d0-3e2f-4b08-aacf-316d9631c006/kube-rbac-proxy-web/0.log" Jul 16 12:50:41.948202 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:41.948132 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_491953d0-3e2f-4b08-aacf-316d9631c006/kube-rbac-proxy/0.log" Jul 16 12:50:41.985645 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:41.985616 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_491953d0-3e2f-4b08-aacf-316d9631c006/kube-rbac-proxy-metric/0.log" Jul 16 12:50:42.010815 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:42.010788 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_491953d0-3e2f-4b08-aacf-316d9631c006/prom-label-proxy/0.log" Jul 16 12:50:42.037100 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:42.037066 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_491953d0-3e2f-4b08-aacf-316d9631c006/init-config-reloader/0.log" Jul 16 12:50:42.194867 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:42.194835 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-74c5dc9c58-wngvz_53c19468-fe34-48fb-b05a-513d02394a55/metrics-server/0.log" Jul 16 12:50:42.230329 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:42.230230 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-5gd8x_27006609-ebbc-4519-a04e-0785af6dadbd/monitoring-plugin/0.log" Jul 16 12:50:42.355451 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:42.355407 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nmpvs_7601f52b-2d35-4c6c-a682-719c85050b81/node-exporter/0.log" Jul 16 12:50:42.380513 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:42.380478 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nmpvs_7601f52b-2d35-4c6c-a682-719c85050b81/kube-rbac-proxy/0.log" Jul 16 12:50:42.410147 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:42.410104 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nmpvs_7601f52b-2d35-4c6c-a682-719c85050b81/init-textfile/0.log" Jul 16 12:50:42.816225 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:42.816131 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-89llm_83e1ab03-695b-4979-8df4-7b647f11b770/prometheus-operator/0.log" Jul 16 12:50:42.837395 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:42.837370 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-89llm_83e1ab03-695b-4979-8df4-7b647f11b770/kube-rbac-proxy/0.log" Jul 16 12:50:42.872280 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:42.872252 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-4vxgp_28f37fb8-7bc6-4d74-a948-b39a0b259637/prometheus-operator-admission-webhook/0.log" Jul 16 12:50:42.982935 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:42.982909 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-58d688cdfc-wrpdp_5091f4e3-046e-4fe9-a173-976eb62cbd62/thanos-query/0.log" Jul 16 12:50:43.010599 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:43.010554 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-58d688cdfc-wrpdp_5091f4e3-046e-4fe9-a173-976eb62cbd62/kube-rbac-proxy-web/0.log" Jul 16 12:50:43.033247 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:43.033221 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-58d688cdfc-wrpdp_5091f4e3-046e-4fe9-a173-976eb62cbd62/kube-rbac-proxy/0.log" Jul 16 12:50:43.061134 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:43.061103 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-58d688cdfc-wrpdp_5091f4e3-046e-4fe9-a173-976eb62cbd62/prom-label-proxy/0.log" Jul 16 12:50:43.085970 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:43.085857 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-58d688cdfc-wrpdp_5091f4e3-046e-4fe9-a173-976eb62cbd62/kube-rbac-proxy-rules/0.log" Jul 16 12:50:43.113845 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:43.113820 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-58d688cdfc-wrpdp_5091f4e3-046e-4fe9-a173-976eb62cbd62/kube-rbac-proxy-metrics/0.log" Jul 16 12:50:44.157494 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.157459 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb"] Jul 16 12:50:44.165376 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.165350 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.172376 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.172347 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb"] Jul 16 12:50:44.322023 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.321993 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fdb974aa-34b9-4520-8f31-48692cf78276-podres\") pod \"perf-node-gather-daemonset-wxsgb\" (UID: \"fdb974aa-34b9-4520-8f31-48692cf78276\") " pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.322023 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.322032 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fdb974aa-34b9-4520-8f31-48692cf78276-lib-modules\") pod \"perf-node-gather-daemonset-wxsgb\" (UID: \"fdb974aa-34b9-4520-8f31-48692cf78276\") " pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.322307 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.322073 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fdb974aa-34b9-4520-8f31-48692cf78276-sys\") pod \"perf-node-gather-daemonset-wxsgb\" (UID: \"fdb974aa-34b9-4520-8f31-48692cf78276\") " pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.322307 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.322199 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fdb974aa-34b9-4520-8f31-48692cf78276-proc\") pod \"perf-node-gather-daemonset-wxsgb\" (UID: \"fdb974aa-34b9-4520-8f31-48692cf78276\") " pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.322307 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.322249 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t49bl\" (UniqueName: \"kubernetes.io/projected/fdb974aa-34b9-4520-8f31-48692cf78276-kube-api-access-t49bl\") pod \"perf-node-gather-daemonset-wxsgb\" (UID: \"fdb974aa-34b9-4520-8f31-48692cf78276\") " pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.423387 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.423295 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fdb974aa-34b9-4520-8f31-48692cf78276-proc\") pod \"perf-node-gather-daemonset-wxsgb\" (UID: \"fdb974aa-34b9-4520-8f31-48692cf78276\") " pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.423601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.423407 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t49bl\" (UniqueName: \"kubernetes.io/projected/fdb974aa-34b9-4520-8f31-48692cf78276-kube-api-access-t49bl\") pod \"perf-node-gather-daemonset-wxsgb\" (UID: \"fdb974aa-34b9-4520-8f31-48692cf78276\") " pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.423601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.423490 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fdb974aa-34b9-4520-8f31-48692cf78276-podres\") pod \"perf-node-gather-daemonset-wxsgb\" (UID: \"fdb974aa-34b9-4520-8f31-48692cf78276\") " pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.423601 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.423548 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fdb974aa-34b9-4520-8f31-48692cf78276-lib-modules\") pod \"perf-node-gather-daemonset-wxsgb\" (UID: \"fdb974aa-34b9-4520-8f31-48692cf78276\") " pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.423783 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.423615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fdb974aa-34b9-4520-8f31-48692cf78276-sys\") pod \"perf-node-gather-daemonset-wxsgb\" (UID: \"fdb974aa-34b9-4520-8f31-48692cf78276\") " pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.423783 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.423727 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fdb974aa-34b9-4520-8f31-48692cf78276-sys\") pod \"perf-node-gather-daemonset-wxsgb\" (UID: \"fdb974aa-34b9-4520-8f31-48692cf78276\") " pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.423903 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.423874 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fdb974aa-34b9-4520-8f31-48692cf78276-podres\") pod \"perf-node-gather-daemonset-wxsgb\" (UID: \"fdb974aa-34b9-4520-8f31-48692cf78276\") " pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.423903 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.423880 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fdb974aa-34b9-4520-8f31-48692cf78276-lib-modules\") pod \"perf-node-gather-daemonset-wxsgb\" (UID: \"fdb974aa-34b9-4520-8f31-48692cf78276\") " pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.424016 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.423948 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fdb974aa-34b9-4520-8f31-48692cf78276-proc\") pod \"perf-node-gather-daemonset-wxsgb\" (UID: \"fdb974aa-34b9-4520-8f31-48692cf78276\") " pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.432517 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.432486 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t49bl\" (UniqueName: \"kubernetes.io/projected/fdb974aa-34b9-4520-8f31-48692cf78276-kube-api-access-t49bl\") pod \"perf-node-gather-daemonset-wxsgb\" (UID: \"fdb974aa-34b9-4520-8f31-48692cf78276\") " pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.479032 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.478996 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.651454 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.651052 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb"] Jul 16 12:50:44.654228 ip-10-0-140-160 kubenswrapper[2578]: W0716 12:50:44.654201 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfdb974aa_34b9_4520_8f31_48692cf78276.slice/crio-f407edae360560272be0f61050d373c5a4f100d0e0933bc6c67800185a04d991 WatchSource:0}: Error finding container f407edae360560272be0f61050d373c5a4f100d0e0933bc6c67800185a04d991: Status 404 returned error can't find the container with id f407edae360560272be0f61050d373c5a4f100d0e0933bc6c67800185a04d991 Jul 16 12:50:44.877588 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.877554 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/1.log" Jul 16 12:50:44.883538 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.883508 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-72p8q_c72b21b6-dfa9-49e6-b546-b0ac30e2216b/console-operator/2.log" Jul 16 12:50:44.941054 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.940963 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" event={"ID":"fdb974aa-34b9-4520-8f31-48692cf78276","Type":"ContainerStarted","Data":"2331a765f91be75e53a4ec459da0db1c38e4f04d7586fd68d6bcae6969355405"} Jul 16 12:50:44.941214 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.941058 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" event={"ID":"fdb974aa-34b9-4520-8f31-48692cf78276","Type":"ContainerStarted","Data":"f407edae360560272be0f61050d373c5a4f100d0e0933bc6c67800185a04d991"} Jul 16 12:50:44.941279 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.941227 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:44.962525 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:44.962464 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" podStartSLOduration=0.962443933 podStartE2EDuration="962.443933ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:50:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:50:44.958151908 +0000 UTC m=+2594.721521383" watchObservedRunningTime="2026-07-16 12:50:44.962443933 +0000 UTC m=+2594.725813395" Jul 16 12:50:45.410191 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:45.410154 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-pvgch_da0d38c7-23cc-4057-bb7a-0de38b2ee970/download-server/0.log" Jul 16 12:50:45.967769 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:45.967731 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-c99fx_1c7ed060-2647-40fb-b395-6654ac05cd42/volume-data-source-validator/0.log" Jul 16 12:50:47.319156 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:47.319129 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qblpk_8174e182-bed0-49f1-bdae-aeaaa73fd83c/dns/0.log" Jul 16 12:50:47.368930 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:47.368904 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qblpk_8174e182-bed0-49f1-bdae-aeaaa73fd83c/kube-rbac-proxy/0.log" Jul 16 12:50:47.504687 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:47.504650 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r7c42_e00ecaab-22f9-4338-ac72-f0eaad408347/dns-node-resolver/0.log" Jul 16 12:50:48.083742 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:48.083711 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-64d797f876-5qg7q_7331897a-45b7-4b04-be5c-9bb16e47b9a9/registry/0.log" Jul 16 12:50:48.202627 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:48.202588 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wjpmx_833615c3-9d95-448d-a709-4735c575a995/node-ca/0.log" Jul 16 12:50:49.295955 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:49.295908 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6lsdz_956ffded-32d6-416b-baca-9de1b5cc5747/discovery/0.log" Jul 16 12:50:49.554009 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:49.553912 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68956567b4-ccm7k_060e6722-ce48-468c-aa89-cc07918db901/router/0.log" Jul 16 12:50:50.251672 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:50.251630 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-knfq6_10ea27e0-362a-4adb-a93c-e3db335e4610/serve-healthcheck-canary/0.log" Jul 16 12:50:50.812365 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:50.812325 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-277mm_f73a8a43-dd26-4e85-865c-a1aba38315c0/insights-operator/0.log" Jul 16 12:50:50.813484 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:50.813457 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-277mm_f73a8a43-dd26-4e85-865c-a1aba38315c0/insights-operator/1.log" Jul 16 12:50:50.958773 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:50.958739 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xm7nj/perf-node-gather-daemonset-wxsgb" Jul 16 12:50:50.981016 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:50.980990 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-znnr6_222026e5-8cb9-43d6-bbe0-25dcb7c66ce6/kube-rbac-proxy/0.log" Jul 16 12:50:51.004279 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:51.004247 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-znnr6_222026e5-8cb9-43d6-bbe0-25dcb7c66ce6/exporter/0.log" Jul 16 12:50:51.025835 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:51.025807 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-znnr6_222026e5-8cb9-43d6-bbe0-25dcb7c66ce6/extractor/0.log" Jul 16 12:50:53.938815 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:53.938780 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm_92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0/manager/0.log" Jul 16 12:50:53.966229 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:53.966197 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fc9bf88fc-mbmmm_92cb28e2-5b64-4acd-b4b5-e2fbfd2ec5e0/copy-manifests/0.log" Jul 16 12:50:54.522905 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:54.522871 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6lsdz_956ffded-32d6-416b-baca-9de1b5cc5747/discovery/0.log" Jul 16 12:50:54.662430 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:54.662393 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68956567b4-ccm7k_060e6722-ce48-468c-aa89-cc07918db901/router/0.log" Jul 16 12:50:55.249494 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:55.249398 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-69c8d477dd-rcnbs_b19e6162-bd6b-4d2b-8917-448e7709a1af/manager/0.log" Jul 16 12:50:55.354238 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:55.354203 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-lqffn_c36a8149-0d59-4753-bf82-af6b60323e61/openshift-lws-operator/0.log" Jul 16 12:50:56.001509 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:56.001476 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6lsdz_956ffded-32d6-416b-baca-9de1b5cc5747/discovery/0.log" Jul 16 12:50:56.254725 ip-10-0-140-160 kubenswrapper[2578]: I0716 12:50:56.254637 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68956567b4-ccm7k_060e6722-ce48-468c-aa89-cc07918db901/router/0.log"