Jul 07 17:25:30.686584 ip-10-0-141-75 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 07 17:25:30.686599 ip-10-0-141-75 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 07 17:25:30.686607 ip-10-0-141-75 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 07 17:25:30.686854 ip-10-0-141-75 systemd[1]: Failed to start Kubernetes Kubelet. Jul 07 17:25:40.938834 ip-10-0-141-75 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 07 17:25:40.938878 ip-10-0-141-75 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e7fc1f1bd0dd4eb7b01126dfb3fed070 -- Jul 07 17:27:22.220791 ip-10-0-141-75 systemd[1]: Starting Kubernetes Kubelet... Jul 07 17:27:22.643531 ip-10-0-141-75 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 17:27:22.643531 ip-10-0-141-75 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 17:27:22.643531 ip-10-0-141-75 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 17:27:22.643531 ip-10-0-141-75 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 17:27:22.643531 ip-10-0-141-75 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 17:27:22.644411 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.644264 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 17:27:22.648720 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648702 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648722 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648728 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648732 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648735 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648738 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648741 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648743 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648746 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648749 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648751 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648754 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648757 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648760 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648762 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648764 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648767 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648769 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648773 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:27:22.648767 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648776 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648780 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648782 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648785 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648788 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648790 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648793 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648803 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648806 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648809 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648811 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648814 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648817 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648819 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648822 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648824 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648827 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648829 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648831 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648834 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:27:22.649214 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648837 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648841 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648843 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648846 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648849 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648852 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648855 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648857 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648860 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648863 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648865 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648868 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648870 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648872 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648875 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648878 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648881 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648883 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648886 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648890 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:27:22.649710 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648893 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648896 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648899 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648902 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648904 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648907 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648909 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648912 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648915 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648917 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648919 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648923 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648926 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648928 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648931 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648933 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648936 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648939 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648941 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648944 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:27:22.650192 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648947 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648950 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648954 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648957 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648960 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648962 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648966 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648969 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648972 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648974 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648977 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648979 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648982 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648985 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648987 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648990 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648992 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648995 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648997 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:27:22.650725 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.648999 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649758 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649765 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649768 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649770 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649773 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649776 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649779 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649781 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649784 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649787 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649789 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649792 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649794 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649798 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649802 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649804 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649807 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649810 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649813 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:27:22.651183 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649816 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649818 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649821 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649823 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649826 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649829 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649831 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649834 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649836 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649839 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649841 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649844 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649847 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649849 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649854 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649858 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649861 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649865 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649868 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:27:22.651682 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649871 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649874 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649877 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649879 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649882 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649884 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649887 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649889 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649892 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649895 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649897 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649900 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649907 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649909 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649912 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649914 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649917 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649920 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649923 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649925 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:27:22.652142 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649928 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649931 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649934 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649936 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649940 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649943 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649946 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649948 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649951 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649955 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649958 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649961 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649964 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649967 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649969 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649974 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649976 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649980 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649983 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:27:22.652635 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649985 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649988 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649991 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649993 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.649996 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650000 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650003 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650005 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650008 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650010 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650012 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650015 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650017 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650020 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650023 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650025 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650028 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650030 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650033 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:27:22.653106 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650036 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650038 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650041 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650413 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650419 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650441 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650445 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650448 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650451 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650454 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650456 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650459 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650462 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650464 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650468 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650471 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650474 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650477 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650481 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650483 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:27:22.653598 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650486 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650489 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650493 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650496 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650499 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650502 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650506 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650509 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650512 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650515 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650518 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650520 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650523 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650525 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650528 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650530 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650533 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650535 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650538 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:27:22.654089 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650540 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650543 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650546 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650549 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650551 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650554 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650556 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650558 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650561 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650565 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650568 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650572 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650575 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650577 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650580 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650582 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650585 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650587 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650590 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:27:22.654566 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650593 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650595 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650598 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650600 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650603 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650605 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650608 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650612 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650614 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650616 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650619 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650621 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650624 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650626 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650629 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650631 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650634 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650636 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650639 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650641 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:27:22.655036 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650644 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650646 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650649 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650651 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650655 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650658 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650660 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650663 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650666 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650668 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650671 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650673 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650675 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650678 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650680 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650683 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650685 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650688 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650690 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:27:22.655563 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650701 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650705 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650708 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650711 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.650714 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650796 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650803 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650813 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650821 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650828 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650833 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650841 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650846 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650850 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650852 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650856 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650859 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650865 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650868 2562 flags.go:64] FLAG: --cgroup-root="" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650871 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650874 2562 flags.go:64] FLAG: --client-ca-file="" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650876 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650879 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650883 2562 flags.go:64] FLAG: --cluster-domain="" Jul 07 17:27:22.656108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650886 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650889 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650894 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650897 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650900 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650903 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650907 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650910 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650913 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650916 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650919 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650922 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650925 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650929 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650932 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650935 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650937 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650940 2562 flags.go:64] FLAG: --enable-server="true" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650943 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650948 2562 flags.go:64] FLAG: --event-burst="100" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650951 2562 flags.go:64] FLAG: --event-qps="50" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650954 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650957 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650960 2562 flags.go:64] FLAG: --eviction-hard="" Jul 07 17:27:22.656698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650963 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650968 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650972 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650975 2562 flags.go:64] FLAG: --eviction-soft="" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650978 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650980 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650984 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650986 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650989 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650992 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.650996 2562 flags.go:64] FLAG: --feature-gates="" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651000 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651003 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651006 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651009 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651012 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651015 2562 flags.go:64] FLAG: --help="false" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651018 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-141-75.ec2.internal" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651021 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651023 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651026 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651029 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651033 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651036 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 17:27:22.657302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651038 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651041 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651044 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651046 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651050 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651052 2562 flags.go:64] FLAG: --kube-reserved="" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651055 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651058 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651061 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651065 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651068 2562 flags.go:64] FLAG: --lock-file="" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651072 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651075 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651079 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651084 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651087 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651090 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651093 2562 flags.go:64] FLAG: --logging-format="text" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651099 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651102 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651105 2562 flags.go:64] FLAG: --manifest-url="" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651108 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651112 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651115 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651119 2562 flags.go:64] FLAG: --max-pods="110" Jul 07 17:27:22.657920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651122 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651125 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651128 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651131 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651134 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651137 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651139 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651162 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651165 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651168 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651171 2562 flags.go:64] FLAG: --pod-cidr="" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651174 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651179 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651182 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651185 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651189 2562 flags.go:64] FLAG: --port="10250" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651192 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651196 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01117c88c0e539cf2" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651199 2562 flags.go:64] FLAG: --qos-reserved="" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651205 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651208 2562 flags.go:64] FLAG: --register-node="true" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651211 2562 flags.go:64] FLAG: --register-with-taints="" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651215 2562 flags.go:64] FLAG: --registry-burst="10" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651217 2562 flags.go:64] FLAG: --registry-qps="5" Jul 07 17:27:22.658540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651220 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651224 2562 flags.go:64] FLAG: --reserved-memory="" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651228 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651231 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651234 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651236 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651239 2562 flags.go:64] FLAG: --runonce="false" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651242 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651245 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651248 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651251 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651254 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651257 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651261 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651264 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651266 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651269 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651272 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651275 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651278 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651280 2562 flags.go:64] FLAG: --system-cgroups="" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651283 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651289 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651292 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651294 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 17:27:22.659171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651299 2562 flags.go:64] FLAG: --tls-min-version="" Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651302 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651305 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651308 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651311 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651314 2562 flags.go:64] FLAG: --v="2" Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651318 2562 flags.go:64] FLAG: --version="false" Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651322 2562 flags.go:64] FLAG: --vmodule="" Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651327 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.651330 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651462 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651466 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651469 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651473 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651476 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651479 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651482 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651484 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651487 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651489 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651492 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651494 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:27:22.659796 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651497 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651499 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651502 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651504 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651507 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651509 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651512 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651514 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651517 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651520 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651524 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651527 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651529 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651532 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651535 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651538 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651542 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651546 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651550 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:27:22.660315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651553 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651556 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651558 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651561 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651563 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651566 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651568 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651571 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651573 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651576 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651579 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651581 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651584 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651586 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651589 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651591 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651594 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651596 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651598 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:27:22.660804 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651601 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651603 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651606 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651608 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651612 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651616 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651620 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651624 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651627 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651630 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651633 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651635 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651639 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651641 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651644 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651646 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651649 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651651 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651654 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:27:22.661266 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651656 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651659 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651661 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651664 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651666 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651668 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651671 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651673 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651676 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651678 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651681 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651683 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651686 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651689 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651692 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651695 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651697 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651701 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651704 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651707 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:27:22.661752 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651709 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:27:22.662226 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651712 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:27:22.662226 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651714 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:27:22.662226 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651717 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:27:22.662226 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651720 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:27:22.662226 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651723 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:27:22.662226 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651726 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:27:22.662226 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651728 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:27:22.662226 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651731 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:27:22.662226 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.651733 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:27:22.662226 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.652271 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 17:27:22.662226 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.658892 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 17:27:22.662226 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.658910 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 17:27:22.662226 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.659608 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 17:27:22.662226 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.659616 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 17:27:22.662226 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659680 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659685 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659689 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659692 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659696 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659700 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659703 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659706 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659709 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659714 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659718 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659721 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659725 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659728 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659731 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659733 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659736 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659738 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659741 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:27:22.662661 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659744 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659746 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659749 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659752 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659754 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659757 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659759 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659762 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659764 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659766 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659771 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659774 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659777 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659779 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659782 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659784 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659787 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659789 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659792 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659794 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:27:22.663137 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659797 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659800 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659802 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659805 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659807 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659810 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659813 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659815 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659818 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659820 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659822 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659825 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659827 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659830 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659832 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659835 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659837 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659840 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659843 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:27:22.663628 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659845 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659847 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659851 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659854 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659857 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659860 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659862 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659865 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659867 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659870 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659872 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659874 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659877 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659879 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659882 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659884 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659887 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659889 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659892 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659894 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659896 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:27:22.664085 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659899 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659902 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659904 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659907 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659910 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659914 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659916 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659919 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659921 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659924 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659926 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659929 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659931 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659934 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659938 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659940 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659943 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659946 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659948 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:27:22.664648 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.659951 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:27:22.665111 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.659956 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 17:27:22.665111 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660097 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:27:22.665111 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660104 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:27:22.665111 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660108 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:27:22.665111 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660110 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:27:22.665111 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660113 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:27:22.665111 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660116 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:27:22.665111 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660118 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:27:22.665111 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660121 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:27:22.665111 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660123 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:27:22.665111 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660126 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:27:22.665111 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660128 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:27:22.665111 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660130 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:27:22.665111 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660133 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660135 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660138 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660140 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660143 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660145 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660148 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660150 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660153 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660157 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660160 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660163 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660165 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660168 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660171 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660173 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660177 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660179 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660182 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660184 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:27:22.665541 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660187 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660189 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660192 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660194 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660197 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660199 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660202 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660204 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660207 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660210 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660212 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660214 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660217 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660220 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660224 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660227 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660230 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660232 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660235 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660237 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:27:22.666042 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660240 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660242 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660245 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660247 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660250 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660252 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660255 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660257 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660259 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660263 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660265 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660267 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660270 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660272 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660275 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660277 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660279 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660282 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660284 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:27:22.666605 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660287 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660289 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660291 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660294 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660296 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660299 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660301 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660304 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660307 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660309 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660311 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660314 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660316 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660319 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660321 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660324 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660327 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660329 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660331 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660334 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:27:22.667861 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660337 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:27:22.668345 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660339 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:27:22.668345 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660341 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:27:22.668345 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660344 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:27:22.668345 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660347 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:27:22.668345 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660349 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:27:22.668345 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660351 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:27:22.668345 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:22.660354 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:27:22.668345 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.660358 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 17:27:22.668345 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.660503 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 17:27:22.668345 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.662403 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 17:27:22.668345 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.663571 2562 server.go:1019] "Starting client certificate rotation" Jul 07 17:27:22.668345 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.663674 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 17:27:22.668345 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.663713 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 17:27:22.668727 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.665676 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 17:27:22.668754 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.668742 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 17:27:22.677136 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.677092 2562 log.go:25] "Validated CRI v1 runtime API" Jul 07 17:27:22.687059 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.687032 2562 log.go:25] "Validated CRI v1 image API" Jul 07 17:27:22.688952 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.688933 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 17:27:22.694234 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.694211 2562 fs.go:135] Filesystem UUIDs: map[38ffa2d0-57d6-49cd-a689-b888829a3f09:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a9b331aa-15af-4819-bae9-61b1b4cc2b5d:/dev/nvme0n1p4] Jul 07 17:27:22.694320 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.694233 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 17:27:22.699269 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.699148 2562 manager.go:216] Machine: {Timestamp:2026-07-07 17:27:22.697953392 +0000 UTC m=+0.348130754 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3234083 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27a938d97be743a66fbb386c515dd6 SystemUUID:ec27a938-d97b-e743-a66f-bb386c515dd6 BootID:e7fc1f1b-d0dd-4eb7-b011-26dfb3fed070 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:66:99:e8:08:07 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:66:99:e8:08:07 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:92:5c:90:3a:0f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 17:27:22.699269 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.699261 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 17:27:22.699405 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.699349 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 17:27:22.700974 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.700944 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 17:27:22.701121 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.700977 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-75.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 17:27:22.701580 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.701569 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 17:27:22.701619 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.701583 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 17:27:22.701619 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.701590 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 17:27:22.701917 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.701898 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 17:27:22.702112 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.702101 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 17:27:22.703130 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.703119 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 17:27:22.705596 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.705583 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 07 17:27:22.705710 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.705700 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 17:27:22.705739 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.705731 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 07 17:27:22.705779 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.705741 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 17:27:22.705779 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.705762 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 17:27:22.705779 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.705775 2562 kubelet.go:398] "Adding apiserver pod source" Jul 07 17:27:22.705863 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.705790 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 17:27:22.706259 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.706246 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 17:27:22.706312 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.706272 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 17:27:22.708338 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.708317 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 17:27:22.709987 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.709974 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 17:27:22.710036 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.710005 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 17:27:22.711346 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.711331 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 17:27:22.711346 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.711348 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 17:27:22.711463 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.711355 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 17:27:22.711463 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.711360 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 17:27:22.711463 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.711369 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 17:27:22.711463 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.711374 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 17:27:22.711463 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.711383 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 17:27:22.711463 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.711388 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 17:27:22.711463 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.711394 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 17:27:22.711463 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.711399 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 17:27:22.711463 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.711408 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 17:27:22.711463 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.711414 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 17:27:22.712125 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.712113 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 17:27:22.712125 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.712125 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 17:27:22.716010 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.715992 2562 server.go:1268] "Started kubelet" Jul 07 17:27:22.716134 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.716084 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 17:27:22.716195 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.716167 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 17:27:22.716311 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.716288 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 17:27:22.716858 ip-10-0-141-75 systemd[1]: Started Kubernetes Kubelet. Jul 07 17:27:22.717919 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.717860 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 17:27:22.718721 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.718689 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-75.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 17:27:22.718925 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.718897 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 17:27:22.719000 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.718959 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-75.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 17:27:22.719903 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.719884 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 07 17:27:22.722845 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.722533 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 17:27:22.723912 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.723891 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 17:27:22.723912 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.723905 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 17:27:22.724119 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.724096 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-75.ec2.internal\" not found" Jul 07 17:27:22.724562 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.724544 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 17:27:22.724562 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.724562 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 17:27:22.724700 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.724617 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 17:27:22.724700 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.724692 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 07 17:27:22.724700 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.724702 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 07 17:27:22.725051 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.725033 2562 factory.go:153] Registering CRI-O factory Jul 07 17:27:22.725169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.725058 2562 factory.go:223] Registration of the crio container factory successfully Jul 07 17:27:22.725169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.725095 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 17:27:22.725169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.725102 2562 factory.go:55] Registering systemd factory Jul 07 17:27:22.725169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.725107 2562 factory.go:223] Registration of the systemd container factory successfully Jul 07 17:27:22.725169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.725122 2562 factory.go:103] Registering Raw factory Jul 07 17:27:22.725169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.725140 2562 manager.go:1196] Started watching for new ooms in manager Jul 07 17:27:22.725677 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.725663 2562 manager.go:349] Starting recovery of all containers Jul 07 17:27:22.726001 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.725065 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-75.ec2.internal.18c011ffc8721aa5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-75.ec2.internal,UID:ip-10-0-141-75.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-75.ec2.internal,},FirstTimestamp:2026-07-07 17:27:22.715970213 +0000 UTC m=+0.366147646,LastTimestamp:2026-07-07 17:27:22.715970213 +0000 UTC m=+0.366147646,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-75.ec2.internal,}" Jul 07 17:27:22.736508 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.736487 2562 manager.go:354] Recovery completed Jul 07 17:27:22.736949 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.736924 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-75.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 17:27:22.737066 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.737044 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 17:27:22.740958 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.740944 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:27:22.742868 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.742839 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:27:22.742959 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.742882 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:27:22.742959 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.742893 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:27:22.743469 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.743448 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 17:27:22.743469 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.743465 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 17:27:22.743581 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.743470 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:27:22.743581 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.743484 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 07 17:27:22.744249 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.744234 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:27:22.744322 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.744264 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:27:22.744322 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.744273 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:27:22.744580 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.744412 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-75.ec2.internal.18c011ffca0c8699 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-75.ec2.internal,UID:ip-10-0-141-75.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-75.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-75.ec2.internal,},FirstTimestamp:2026-07-07 17:27:22.742867609 +0000 UTC m=+0.393044978,LastTimestamp:2026-07-07 17:27:22.742867609 +0000 UTC m=+0.393044978,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-75.ec2.internal,}" Jul 07 17:27:22.744634 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.744584 2562 policy_none.go:49] "None policy: Start" Jul 07 17:27:22.744634 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.744599 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 17:27:22.744634 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.744612 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 17:27:22.745464 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.745453 2562 policy_none.go:47] "Start" Jul 07 17:27:22.754190 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.754020 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-75.ec2.internal.18c011ffca0cd28e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-75.ec2.internal,UID:ip-10-0-141-75.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-75.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-75.ec2.internal,},FirstTimestamp:2026-07-07 17:27:22.742887054 +0000 UTC m=+0.393064416,LastTimestamp:2026-07-07 17:27:22.742887054 +0000 UTC m=+0.393064416,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-75.ec2.internal,}" Jul 07 17:27:22.756714 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.756644 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-75.ec2.internal.18c011ffca0cf7f3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-75.ec2.internal,UID:ip-10-0-141-75.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-75.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-75.ec2.internal,},FirstTimestamp:2026-07-07 17:27:22.742896627 +0000 UTC m=+0.393073990,LastTimestamp:2026-07-07 17:27:22.742896627 +0000 UTC m=+0.393073990,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-75.ec2.internal,}" Jul 07 17:27:22.769313 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.769214 2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-75.ec2.internal.18c011ffca0c8699\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-75.ec2.internal.18c011ffca0c8699 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-75.ec2.internal,UID:ip-10-0-141-75.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-75.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-75.ec2.internal,},FirstTimestamp:2026-07-07 17:27:22.742867609 +0000 UTC m=+0.393044978,LastTimestamp:2026-07-07 17:27:22.744247853 +0000 UTC m=+0.394425219,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-75.ec2.internal,}" Jul 07 17:27:22.779661 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.779565 2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-75.ec2.internal.18c011ffca0cd28e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-75.ec2.internal.18c011ffca0cd28e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-75.ec2.internal,UID:ip-10-0-141-75.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-75.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-75.ec2.internal,},FirstTimestamp:2026-07-07 17:27:22.742887054 +0000 UTC m=+0.393064416,LastTimestamp:2026-07-07 17:27:22.7442685 +0000 UTC m=+0.394445865,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-75.ec2.internal,}" Jul 07 17:27:22.786394 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.786373 2562 manager.go:337] "Starting Device Plugin manager" Jul 07 17:27:22.799011 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.786418 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 17:27:22.799011 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.786467 2562 server.go:87] "Starting device plugin registration server" Jul 07 17:27:22.799011 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.786785 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 17:27:22.799011 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.786803 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 17:27:22.799011 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.786886 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 17:27:22.799011 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.786985 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 17:27:22.799011 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.786998 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 17:27:22.799011 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.787590 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 17:27:22.799011 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.787641 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-75.ec2.internal\" not found" Jul 07 17:27:22.799011 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.792515 2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-75.ec2.internal.18c011ffca0cf7f3\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-75.ec2.internal.18c011ffca0cf7f3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-75.ec2.internal,UID:ip-10-0-141-75.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-75.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-75.ec2.internal,},FirstTimestamp:2026-07-07 17:27:22.742896627 +0000 UTC m=+0.393073990,LastTimestamp:2026-07-07 17:27:22.744276945 +0000 UTC m=+0.394454309,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-75.ec2.internal,}" Jul 07 17:27:22.802035 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.801955 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-75.ec2.internal.18c011ffccca1303 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-75.ec2.internal,UID:ip-10-0-141-75.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-141-75.ec2.internal,},FirstTimestamp:2026-07-07 17:27:22.788844291 +0000 UTC m=+0.439021641,LastTimestamp:2026-07-07 17:27:22.788844291 +0000 UTC m=+0.439021641,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-75.ec2.internal,}" Jul 07 17:27:22.810663 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.810643 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2p2jh" Jul 07 17:27:22.817928 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.817907 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2p2jh" Jul 07 17:27:22.861289 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.861232 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 17:27:22.862723 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.862690 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 17:27:22.862818 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.862754 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 17:27:22.862818 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.862783 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 17:27:22.862896 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.862821 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 17:27:22.870078 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.870053 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 17:27:22.887453 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.887409 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:27:22.888819 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.888793 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:27:22.888926 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.888834 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:27:22.888926 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.888845 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:27:22.888926 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.888868 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-75.ec2.internal" Jul 07 17:27:22.901608 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.901550 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-75.ec2.internal" Jul 07 17:27:22.901608 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.901578 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-75.ec2.internal\": node \"ip-10-0-141-75.ec2.internal\" not found" Jul 07 17:27:22.928273 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.928243 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-75.ec2.internal\" not found" Jul 07 17:27:22.962954 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.962925 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-75.ec2.internal"] Jul 07 17:27:22.963054 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.963022 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:27:22.964073 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.964052 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:27:22.964188 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.964088 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:27:22.964188 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.964101 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:27:22.965318 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.965302 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:27:22.965481 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.965464 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal" Jul 07 17:27:22.965528 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.965502 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:27:22.966134 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.966115 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:27:22.966233 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.966145 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:27:22.966233 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.966119 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:27:22.966233 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.966179 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:27:22.966233 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.966197 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:27:22.966233 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.966155 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:27:22.967188 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.967174 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-75.ec2.internal" Jul 07 17:27:22.967234 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.967200 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:27:22.967951 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.967936 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:27:22.968025 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.967964 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:27:22.968025 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:22.967978 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:27:22.985105 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.985080 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-75.ec2.internal\" not found" node="ip-10-0-141-75.ec2.internal" Jul 07 17:27:22.988815 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:22.988797 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-75.ec2.internal\" not found" node="ip-10-0-141-75.ec2.internal" Jul 07 17:27:23.026963 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.026927 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61717e98e5e11efe1087e5f0939585b1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal\" (UID: \"61717e98e5e11efe1087e5f0939585b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal" Jul 07 17:27:23.026963 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.026965 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61717e98e5e11efe1087e5f0939585b1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal\" (UID: \"61717e98e5e11efe1087e5f0939585b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal" Jul 07 17:27:23.027156 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.026989 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5e37f1de7e940fc5cfbd6ca0135ad2aa-config\") pod \"kube-apiserver-proxy-ip-10-0-141-75.ec2.internal\" (UID: \"5e37f1de7e940fc5cfbd6ca0135ad2aa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-75.ec2.internal" Jul 07 17:27:23.029051 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:23.029028 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-75.ec2.internal\" not found" Jul 07 17:27:23.127893 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.127855 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61717e98e5e11efe1087e5f0939585b1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal\" (UID: \"61717e98e5e11efe1087e5f0939585b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal" Jul 07 17:27:23.127893 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.127889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61717e98e5e11efe1087e5f0939585b1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal\" (UID: \"61717e98e5e11efe1087e5f0939585b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal" Jul 07 17:27:23.128078 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.127909 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5e37f1de7e940fc5cfbd6ca0135ad2aa-config\") pod \"kube-apiserver-proxy-ip-10-0-141-75.ec2.internal\" (UID: \"5e37f1de7e940fc5cfbd6ca0135ad2aa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-75.ec2.internal" Jul 07 17:27:23.128078 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.127969 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5e37f1de7e940fc5cfbd6ca0135ad2aa-config\") pod \"kube-apiserver-proxy-ip-10-0-141-75.ec2.internal\" (UID: \"5e37f1de7e940fc5cfbd6ca0135ad2aa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-75.ec2.internal" Jul 07 17:27:23.128078 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.127982 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61717e98e5e11efe1087e5f0939585b1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal\" (UID: \"61717e98e5e11efe1087e5f0939585b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal" Jul 07 17:27:23.128078 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.127969 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61717e98e5e11efe1087e5f0939585b1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal\" (UID: \"61717e98e5e11efe1087e5f0939585b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal" Jul 07 17:27:23.129969 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:23.129950 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-75.ec2.internal\" not found" Jul 07 17:27:23.230985 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:23.230900 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-75.ec2.internal\" not found" Jul 07 17:27:23.288097 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.288057 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal" Jul 07 17:27:23.292381 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.292086 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-75.ec2.internal" Jul 07 17:27:23.331844 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:23.331809 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-75.ec2.internal\" not found" Jul 07 17:27:23.432385 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:23.432348 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-75.ec2.internal\" not found" Jul 07 17:27:23.533047 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:23.532963 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-75.ec2.internal\" not found" Jul 07 17:27:23.633498 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:23.633463 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-75.ec2.internal\" not found" Jul 07 17:27:23.663640 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.663605 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 17:27:23.664113 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.663762 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 17:27:23.733882 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:23.733849 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-75.ec2.internal\" not found" Jul 07 17:27:23.778681 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:23.778641 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61717e98e5e11efe1087e5f0939585b1.slice/crio-7bb936b800fb64cbbc5b43b07a6dfbb89e3dc3d9e0825f781cc954a336040d2c WatchSource:0}: Error finding container 7bb936b800fb64cbbc5b43b07a6dfbb89e3dc3d9e0825f781cc954a336040d2c: Status 404 returned error can't find the container with id 7bb936b800fb64cbbc5b43b07a6dfbb89e3dc3d9e0825f781cc954a336040d2c Jul 07 17:27:23.778962 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:23.778941 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e37f1de7e940fc5cfbd6ca0135ad2aa.slice/crio-dece8420ed64687f66ba74e88432d902e06054b04973ca5189db12f72766c836 WatchSource:0}: Error finding container dece8420ed64687f66ba74e88432d902e06054b04973ca5189db12f72766c836: Status 404 returned error can't find the container with id dece8420ed64687f66ba74e88432d902e06054b04973ca5189db12f72766c836 Jul 07 17:27:23.784945 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.784930 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 17:27:23.790791 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.790773 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:27:23.791815 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.791797 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:27:23.791906 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.791832 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:27:23.791906 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.791846 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:27:23.820209 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.820153 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 17:22:24 +0000 UTC" deadline="2028-03-20 17:55:26.565644542 +0000 UTC" Jul 07 17:27:23.820209 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.820201 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14928h28m2.745446706s" Jul 07 17:27:23.834744 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:23.834709 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-75.ec2.internal\" not found" Jul 07 17:27:23.866664 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.866608 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal" event={"ID":"61717e98e5e11efe1087e5f0939585b1","Type":"ContainerStarted","Data":"7bb936b800fb64cbbc5b43b07a6dfbb89e3dc3d9e0825f781cc954a336040d2c"} Jul 07 17:27:23.867646 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:23.867625 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-75.ec2.internal" event={"ID":"5e37f1de7e940fc5cfbd6ca0135ad2aa","Type":"ContainerStarted","Data":"dece8420ed64687f66ba74e88432d902e06054b04973ca5189db12f72766c836"} Jul 07 17:27:23.935170 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:23.935139 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-75.ec2.internal\" not found" Jul 07 17:27:24.022861 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:24.022829 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 17:27:24.035548 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:24.035488 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-75.ec2.internal\" not found" Jul 07 17:27:24.047753 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:24.047730 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 17:27:24.116256 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:24.116226 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 17:27:24.124669 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:24.124650 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal" Jul 07 17:27:24.152444 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:24.152403 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 17:27:24.153385 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:24.153370 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-75.ec2.internal" Jul 07 17:27:24.173942 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:24.173916 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 17:27:26.476027 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.475987 2562 apiserver.go:52] "Watching apiserver" Jul 07 17:27:26.484086 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.484052 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 17:27:26.484466 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.484442 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-pc8dw","openshift-network-operator/iptables-alerter-sxhbx","openshift-ovn-kubernetes/ovnkube-node-xzssf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v","openshift-cluster-node-tuning-operator/tuned-zs8ks","openshift-image-registry/node-ca-fx768","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal","openshift-multus/multus-bpm6b","openshift-multus/network-metrics-daemon-7mtf5","openshift-network-diagnostics/network-check-target-47448","kube-system/global-pull-secret-syncer-5nf29","kube-system/konnectivity-agent-77mtq","kube-system/kube-apiserver-proxy-ip-10-0-141-75.ec2.internal","openshift-dns/node-resolver-hsg2d"] Jul 07 17:27:26.485715 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.485693 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:26.485799 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:26.485768 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:26.486907 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.486892 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sxhbx" Jul 07 17:27:26.488222 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.488195 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.489401 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.489377 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.490242 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.490001 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:27:26.490242 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.490029 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4x22v\"" Jul 07 17:27:26.490242 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.490051 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 17:27:26.490242 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.490029 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 17:27:26.490667 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.490650 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.491153 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.491089 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 17:27:26.491254 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.491157 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 17:27:26.491310 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.491279 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-g7tx6\"" Jul 07 17:27:26.491310 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.491125 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 17:27:26.491557 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.491538 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 17:27:26.491676 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.491587 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 17:27:26.491676 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.491654 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 17:27:26.491925 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.491910 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-77mtq" Jul 07 17:27:26.492812 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.492701 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 17:27:26.492812 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.492729 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 17:27:26.492812 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.492788 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cpgqs\"" Jul 07 17:27:26.492812 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.492790 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 17:27:26.493208 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.493193 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:26.493291 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:26.493268 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:26.494153 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.494127 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 17:27:26.495528 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.495506 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:27:26.495618 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.495567 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-c5sft\"" Jul 07 17:27:26.496445 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.496426 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 17:27:26.496648 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.496629 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 17:27:26.496722 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.496706 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 17:27:26.498381 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.497139 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lvv4d\"" Jul 07 17:27:26.498381 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.497206 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.499312 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.499292 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:26.499403 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:26.499366 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:26.500373 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.500352 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 17:27:26.500501 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.500446 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 17:27:26.500634 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.500606 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 17:27:26.500712 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.500656 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 17:27:26.500712 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.500681 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 17:27:26.500895 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.500873 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fx768" Jul 07 17:27:26.501074 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.501024 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-77b67\"" Jul 07 17:27:26.502350 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.502327 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.504237 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.503214 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hsg2d" Jul 07 17:27:26.504237 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.503879 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 17:27:26.504237 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.503912 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 17:27:26.504237 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.504134 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 17:27:26.504237 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.504168 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jc4b7\"" Jul 07 17:27:26.505079 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.504958 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wpxqm\"" Jul 07 17:27:26.505079 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.504991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd6b6\" (UniqueName: \"kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6\") pod \"network-check-target-47448\" (UID: \"688bff8d-e98a-4a83-a526-18eab4e56743\") " pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:26.505079 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505024 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-kubelet\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.505079 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505050 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-cni-bin\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.505079 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.504981 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 17:27:26.505292 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505074 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84v5r\" (UniqueName: \"kubernetes.io/projected/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-kube-api-access-84v5r\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.505292 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505149 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4qlx\" (UniqueName: \"kubernetes.io/projected/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-kube-api-access-v4qlx\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.505292 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505174 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c-iptables-alerter-script\") pod \"iptables-alerter-sxhbx\" (UID: \"1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c\") " pod="openshift-network-operator/iptables-alerter-sxhbx" Jul 07 17:27:26.505292 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505194 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.505292 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505210 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-run-netns\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.505292 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505225 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-var-lib-openvswitch\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.505292 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505265 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-run-ovn-kubernetes\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.505535 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505308 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-etc-selinux\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.505535 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505335 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-sys-fs\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.505535 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505358 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh62f\" (UniqueName: \"kubernetes.io/projected/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-kube-api-access-sh62f\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.505535 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505392 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.505535 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505420 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.505535 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505441 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-socket-dir\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.505535 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505469 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-kubernetes\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.505535 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505497 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-systemd\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.505535 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505519 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-host\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.505535 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505538 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7de1b8d9-18e1-4c4d-9202-fa628f48881d-ovn-node-metrics-cert\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.506000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505553 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-tmp\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.506000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505572 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-cnibin\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.506000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505607 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzmtf\" (UniqueName: \"kubernetes.io/projected/1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c-kube-api-access-lzmtf\") pod \"iptables-alerter-sxhbx\" (UID: \"1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c\") " pod="openshift-network-operator/iptables-alerter-sxhbx" Jul 07 17:27:26.506000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505638 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-slash\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.506000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505660 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-cni-netd\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.506000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505681 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7de1b8d9-18e1-4c4d-9202-fa628f48881d-ovnkube-script-lib\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.506000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505703 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-registration-dir\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.506000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505725 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-sysconfig\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.506000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505738 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-os-release\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.506000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505752 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.506000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505767 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-modprobe-d\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.506000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505787 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-var-lib-kubelet\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.506000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505822 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.506000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505836 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs\") pod \"network-metrics-daemon-7mtf5\" (UID: \"3e184935-aec2-42e9-8944-5199f4d6c1b9\") " pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:26.506000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505883 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f4n4\" (UniqueName: \"kubernetes.io/projected/3e184935-aec2-42e9-8944-5199f4d6c1b9-kube-api-access-4f4n4\") pod \"network-metrics-daemon-7mtf5\" (UID: \"3e184935-aec2-42e9-8944-5199f4d6c1b9\") " pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:26.506000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505912 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-node-log\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505928 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9zb2\" (UniqueName: \"kubernetes.io/projected/7de1b8d9-18e1-4c4d-9202-fa628f48881d-kube-api-access-j9zb2\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505946 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.505992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-device-dir\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506014 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-sysctl-conf\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506034 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-lib-modules\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506059 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c-host-slash\") pod \"iptables-alerter-sxhbx\" (UID: \"1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c\") " pod="openshift-network-operator/iptables-alerter-sxhbx" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506103 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-run-ovn\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506137 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-system-cni-dir\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506160 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506163 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-run-systemd\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506188 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-run-openvswitch\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506215 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-log-socket\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506226 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506239 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-run\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506264 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jrrv4\"" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506292 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-tuned\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506320 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-etc-openvswitch\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.506713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506347 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-sysctl-d\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.507486 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506372 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/55f38dc4-f679-4b32-8f4e-6c3313461f0c-agent-certs\") pod \"konnectivity-agent-77mtq\" (UID: \"55f38dc4-f679-4b32-8f4e-6c3313461f0c\") " pod="kube-system/konnectivity-agent-77mtq" Jul 07 17:27:26.507486 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506397 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7de1b8d9-18e1-4c4d-9202-fa628f48881d-ovnkube-config\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.507486 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506421 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7de1b8d9-18e1-4c4d-9202-fa628f48881d-env-overrides\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.507486 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506455 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-sys\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.507486 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506493 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/55f38dc4-f679-4b32-8f4e-6c3313461f0c-konnectivity-ca\") pod \"konnectivity-agent-77mtq\" (UID: \"55f38dc4-f679-4b32-8f4e-6c3313461f0c\") " pod="kube-system/konnectivity-agent-77mtq" Jul 07 17:27:26.507486 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.506511 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-systemd-units\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.510187 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.510126 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 17:27:26.525926 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.525904 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 17:27:26.549337 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.549310 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2r2lj" Jul 07 17:27:26.559631 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.559604 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2r2lj" Jul 07 17:27:26.594309 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.594277 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 17:27:26.607554 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607516 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/55f38dc4-f679-4b32-8f4e-6c3313461f0c-agent-certs\") pod \"konnectivity-agent-77mtq\" (UID: \"55f38dc4-f679-4b32-8f4e-6c3313461f0c\") " pod="kube-system/konnectivity-agent-77mtq" Jul 07 17:27:26.607740 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607567 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-hostroot\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.607740 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607587 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dfec4ce5-3bf7-4600-b522-9daca21cf8c8-hosts-file\") pod \"node-resolver-hsg2d\" (UID: \"dfec4ce5-3bf7-4600-b522-9daca21cf8c8\") " pod="openshift-dns/node-resolver-hsg2d" Jul 07 17:27:26.607740 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607612 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7de1b8d9-18e1-4c4d-9202-fa628f48881d-ovnkube-config\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.607740 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607628 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7de1b8d9-18e1-4c4d-9202-fa628f48881d-env-overrides\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.607740 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607644 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-sys\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.607740 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607673 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/55f38dc4-f679-4b32-8f4e-6c3313461f0c-konnectivity-ca\") pod \"konnectivity-agent-77mtq\" (UID: \"55f38dc4-f679-4b32-8f4e-6c3313461f0c\") " pod="kube-system/konnectivity-agent-77mtq" Jul 07 17:27:26.607740 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607699 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-run-k8s-cni-cncf-io\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.607740 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607727 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-systemd-units\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607750 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79786501-041d-4306-a959-b11ecc7e71d6-cni-binary-copy\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607773 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-multus-conf-dir\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607794 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtzkl\" (UniqueName: \"kubernetes.io/projected/79786501-041d-4306-a959-b11ecc7e71d6-kube-api-access-dtzkl\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607819 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd6b6\" (UniqueName: \"kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6\") pod \"network-check-target-47448\" (UID: \"688bff8d-e98a-4a83-a526-18eab4e56743\") " pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607845 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-kubelet\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607878 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-cni-bin\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607894 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-os-release\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607909 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-run-netns\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607926 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/658cefc3-6811-4b18-b3a2-3af9ac357158-dbus\") pod \"global-pull-secret-syncer-5nf29\" (UID: \"658cefc3-6811-4b18-b3a2-3af9ac357158\") " pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.607952 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84v5r\" (UniqueName: \"kubernetes.io/projected/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-kube-api-access-84v5r\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608000 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79786501-041d-4306-a959-b11ecc7e71d6-multus-daemon-config\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608026 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86rpn\" (UniqueName: \"kubernetes.io/projected/dfec4ce5-3bf7-4600-b522-9daca21cf8c8-kube-api-access-86rpn\") pod \"node-resolver-hsg2d\" (UID: \"dfec4ce5-3bf7-4600-b522-9daca21cf8c8\") " pod="openshift-dns/node-resolver-hsg2d" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608052 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4qlx\" (UniqueName: \"kubernetes.io/projected/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-kube-api-access-v4qlx\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608078 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c-iptables-alerter-script\") pod \"iptables-alerter-sxhbx\" (UID: \"1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c\") " pod="openshift-network-operator/iptables-alerter-sxhbx" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608096 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-cnibin\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608122 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.608159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608147 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nkl6\" (UniqueName: \"kubernetes.io/projected/84115d28-35c1-4c55-9d01-c0f85ccfda09-kube-api-access-6nkl6\") pod \"node-ca-fx768\" (UID: \"84115d28-35c1-4c55-9d01-c0f85ccfda09\") " pod="openshift-image-registry/node-ca-fx768" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608174 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-run-netns\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608201 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-var-lib-openvswitch\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608226 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-run-ovn-kubernetes\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608251 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-etc-selinux\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608270 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-sys-fs\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608290 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh62f\" (UniqueName: \"kubernetes.io/projected/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-kube-api-access-sh62f\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608307 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608336 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608370 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-socket-dir\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608394 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-kubernetes\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608417 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-systemd\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608443 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-host\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608465 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-system-cni-dir\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608487 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84115d28-35c1-4c55-9d01-c0f85ccfda09-serviceca\") pod \"node-ca-fx768\" (UID: \"84115d28-35c1-4c55-9d01-c0f85ccfda09\") " pod="openshift-image-registry/node-ca-fx768" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608509 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7de1b8d9-18e1-4c4d-9202-fa628f48881d-ovn-node-metrics-cert\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608535 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-tmp\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.608927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608561 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-cnibin\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608592 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzmtf\" (UniqueName: \"kubernetes.io/projected/1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c-kube-api-access-lzmtf\") pod \"iptables-alerter-sxhbx\" (UID: \"1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c\") " pod="openshift-network-operator/iptables-alerter-sxhbx" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608609 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-multus-socket-dir-parent\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608623 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-etc-kubernetes\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608636 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84115d28-35c1-4c55-9d01-c0f85ccfda09-host\") pod \"node-ca-fx768\" (UID: \"84115d28-35c1-4c55-9d01-c0f85ccfda09\") " pod="openshift-image-registry/node-ca-fx768" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608653 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-slash\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608671 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-cni-netd\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608688 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7de1b8d9-18e1-4c4d-9202-fa628f48881d-ovnkube-script-lib\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608704 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-registration-dir\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608730 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-sysconfig\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608755 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-os-release\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608783 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608800 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-modprobe-d\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608816 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-var-lib-kubelet\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608834 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608858 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs\") pod \"network-metrics-daemon-7mtf5\" (UID: \"3e184935-aec2-42e9-8944-5199f4d6c1b9\") " pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608888 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f4n4\" (UniqueName: \"kubernetes.io/projected/3e184935-aec2-42e9-8944-5199f4d6c1b9-kube-api-access-4f4n4\") pod \"network-metrics-daemon-7mtf5\" (UID: \"3e184935-aec2-42e9-8944-5199f4d6c1b9\") " pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:26.609714 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608917 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-multus-cni-dir\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608943 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-var-lib-kubelet\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.608986 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-node-log\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609015 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9zb2\" (UniqueName: \"kubernetes.io/projected/7de1b8d9-18e1-4c4d-9202-fa628f48881d-kube-api-access-j9zb2\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609042 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-device-dir\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609077 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-sysctl-conf\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609094 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-lib-modules\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609110 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c-host-slash\") pod \"iptables-alerter-sxhbx\" (UID: \"1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c\") " pod="openshift-network-operator/iptables-alerter-sxhbx" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609128 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-run-ovn\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609144 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-system-cni-dir\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609161 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-var-lib-cni-bin\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609179 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-var-lib-cni-multus\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609195 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfec4ce5-3bf7-4600-b522-9daca21cf8c8-tmp-dir\") pod \"node-resolver-hsg2d\" (UID: \"dfec4ce5-3bf7-4600-b522-9daca21cf8c8\") " pod="openshift-dns/node-resolver-hsg2d" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609211 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-run-systemd\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609229 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-run-openvswitch\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609264 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-log-socket\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.610391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609282 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-run\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.611196 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609302 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-tuned\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.611196 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609326 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-run-multus-certs\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.611196 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609348 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/658cefc3-6811-4b18-b3a2-3af9ac357158-kubelet-config\") pod \"global-pull-secret-syncer-5nf29\" (UID: \"658cefc3-6811-4b18-b3a2-3af9ac357158\") " pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:26.611196 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609370 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret\") pod \"global-pull-secret-syncer-5nf29\" (UID: \"658cefc3-6811-4b18-b3a2-3af9ac357158\") " pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:26.611196 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609388 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-etc-openvswitch\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.611196 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609414 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-sysctl-d\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.611196 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609581 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-sysctl-d\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.611196 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.609992 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 17:27:26.611196 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.610748 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-cnibin\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.611196 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611036 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7de1b8d9-18e1-4c4d-9202-fa628f48881d-ovnkube-config\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.611705 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611203 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-sys\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.611705 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611338 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.611705 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611393 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-device-dir\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.611705 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611494 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-sysctl-conf\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.611705 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611513 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7de1b8d9-18e1-4c4d-9202-fa628f48881d-env-overrides\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.611705 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611584 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-lib-modules\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.611705 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611610 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-slash\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.611705 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611630 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c-host-slash\") pod \"iptables-alerter-sxhbx\" (UID: \"1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c\") " pod="openshift-network-operator/iptables-alerter-sxhbx" Jul 07 17:27:26.611705 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611660 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-cni-netd\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.611705 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611675 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-run-ovn\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.611705 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611701 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/55f38dc4-f679-4b32-8f4e-6c3313461f0c-konnectivity-ca\") pod \"konnectivity-agent-77mtq\" (UID: \"55f38dc4-f679-4b32-8f4e-6c3313461f0c\") " pod="kube-system/konnectivity-agent-77mtq" Jul 07 17:27:26.612254 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611719 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-system-cni-dir\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.612254 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611767 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-systemd-units\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.612254 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611792 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-run-systemd\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.612254 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611846 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-run-openvswitch\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.612254 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611856 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-run\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.612254 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611904 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-log-socket\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.612254 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.611950 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-kubelet\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.612254 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.612015 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-cni-bin\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.612254 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.612163 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7de1b8d9-18e1-4c4d-9202-fa628f48881d-ovnkube-script-lib\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.612254 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.612208 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-sys-fs\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.612254 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.612227 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-registration-dir\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.612663 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:26.612471 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:27:26.612663 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:26.612563 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs podName:3e184935-aec2-42e9-8944-5199f4d6c1b9 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:27.112519035 +0000 UTC m=+2.994155558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs") pod "network-metrics-daemon-7mtf5" (UID: "3e184935-aec2-42e9-8944-5199f4d6c1b9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:27:26.612810 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.612788 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.612912 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.612893 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.612946 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.612922 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-run-netns\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.613020 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613004 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-run-ovn-kubernetes\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.613070 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613044 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-node-log\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.613070 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613041 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-kubernetes\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.613160 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613144 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-etc-selinux\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613526 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-modprobe-d\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613592 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-var-lib-kubelet\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613631 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c-iptables-alerter-script\") pod \"iptables-alerter-sxhbx\" (UID: \"1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c\") " pod="openshift-network-operator/iptables-alerter-sxhbx" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613639 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-sysconfig\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613668 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-var-lib-openvswitch\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613708 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613714 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-os-release\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613793 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-systemd\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613820 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613843 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-host\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613853 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613925 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7de1b8d9-18e1-4c4d-9202-fa628f48881d-etc-openvswitch\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.613921 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-socket-dir\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.614419 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/55f38dc4-f679-4b32-8f4e-6c3313461f0c-agent-certs\") pod \"konnectivity-agent-77mtq\" (UID: \"55f38dc4-f679-4b32-8f4e-6c3313461f0c\") " pod="kube-system/konnectivity-agent-77mtq" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.615835 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-etc-tuned\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.616690 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7de1b8d9-18e1-4c4d-9202-fa628f48881d-ovn-node-metrics-cert\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.619994 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.617253 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-tmp\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.626631 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:26.626021 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:27:26.626631 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:26.626051 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:27:26.626631 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:26.626066 2562 projected.go:196] Error preparing data for projected volume kube-api-access-sd6b6 for pod openshift-network-diagnostics/network-check-target-47448: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:27:26.626631 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:26.626168 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6 podName:688bff8d-e98a-4a83-a526-18eab4e56743 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:27.126148547 +0000 UTC m=+3.007785069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sd6b6" (UniqueName: "kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6") pod "network-check-target-47448" (UID: "688bff8d-e98a-4a83-a526-18eab4e56743") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:27:26.627644 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.627618 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzmtf\" (UniqueName: \"kubernetes.io/projected/1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c-kube-api-access-lzmtf\") pod \"iptables-alerter-sxhbx\" (UID: \"1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c\") " pod="openshift-network-operator/iptables-alerter-sxhbx" Jul 07 17:27:26.629343 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.629298 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9zb2\" (UniqueName: \"kubernetes.io/projected/7de1b8d9-18e1-4c4d-9202-fa628f48881d-kube-api-access-j9zb2\") pod \"ovnkube-node-xzssf\" (UID: \"7de1b8d9-18e1-4c4d-9202-fa628f48881d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.630713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.630688 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f4n4\" (UniqueName: \"kubernetes.io/projected/3e184935-aec2-42e9-8944-5199f4d6c1b9-kube-api-access-4f4n4\") pod \"network-metrics-daemon-7mtf5\" (UID: \"3e184935-aec2-42e9-8944-5199f4d6c1b9\") " pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:26.631995 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.631595 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh62f\" (UniqueName: \"kubernetes.io/projected/3d96c74b-d6d5-488d-94f7-ad264ce0c44f-kube-api-access-sh62f\") pod \"tuned-zs8ks\" (UID: \"3d96c74b-d6d5-488d-94f7-ad264ce0c44f\") " pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.632750 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.632732 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84v5r\" (UniqueName: \"kubernetes.io/projected/f81d1bd5-43a4-476a-b1c8-2a637c464bc9-kube-api-access-84v5r\") pod \"multus-additional-cni-plugins-pc8dw\" (UID: \"f81d1bd5-43a4-476a-b1c8-2a637c464bc9\") " pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.632864 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.632845 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4qlx\" (UniqueName: \"kubernetes.io/projected/bfe12642-5f4d-478a-8cf5-1c8928ad0f4d-kube-api-access-v4qlx\") pod \"aws-ebs-csi-driver-node-fvg5v\" (UID: \"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.710574 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710526 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-multus-cni-dir\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.710574 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710577 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-var-lib-kubelet\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.710802 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710610 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-var-lib-cni-bin\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.710802 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710634 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-var-lib-cni-multus\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.710802 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710659 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfec4ce5-3bf7-4600-b522-9daca21cf8c8-tmp-dir\") pod \"node-resolver-hsg2d\" (UID: \"dfec4ce5-3bf7-4600-b522-9daca21cf8c8\") " pod="openshift-dns/node-resolver-hsg2d" Jul 07 17:27:26.710802 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710689 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-run-multus-certs\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.710802 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710713 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/658cefc3-6811-4b18-b3a2-3af9ac357158-kubelet-config\") pod \"global-pull-secret-syncer-5nf29\" (UID: \"658cefc3-6811-4b18-b3a2-3af9ac357158\") " pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:26.710802 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710713 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-var-lib-kubelet\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.710802 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710736 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret\") pod \"global-pull-secret-syncer-5nf29\" (UID: \"658cefc3-6811-4b18-b3a2-3af9ac357158\") " pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:26.710802 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710778 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-hostroot\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.710802 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710798 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dfec4ce5-3bf7-4600-b522-9daca21cf8c8-hosts-file\") pod \"node-resolver-hsg2d\" (UID: \"dfec4ce5-3bf7-4600-b522-9daca21cf8c8\") " pod="openshift-dns/node-resolver-hsg2d" Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:26.710811 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:26.710855 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret podName:658cefc3-6811-4b18-b3a2-3af9ac357158 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:27.210840803 +0000 UTC m=+3.092477313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret") pod "global-pull-secret-syncer-5nf29" (UID: "658cefc3-6811-4b18-b3a2-3af9ac357158") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710875 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-run-k8s-cni-cncf-io\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710891 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79786501-041d-4306-a959-b11ecc7e71d6-cni-binary-copy\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710907 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-multus-conf-dir\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710922 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtzkl\" (UniqueName: \"kubernetes.io/projected/79786501-041d-4306-a959-b11ecc7e71d6-kube-api-access-dtzkl\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710929 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dfec4ce5-3bf7-4600-b522-9daca21cf8c8-hosts-file\") pod \"node-resolver-hsg2d\" (UID: \"dfec4ce5-3bf7-4600-b522-9daca21cf8c8\") " pod="openshift-dns/node-resolver-hsg2d" Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710723 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-multus-cni-dir\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-os-release\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710961 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-run-netns\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710984 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfec4ce5-3bf7-4600-b522-9daca21cf8c8-tmp-dir\") pod \"node-resolver-hsg2d\" (UID: \"dfec4ce5-3bf7-4600-b522-9daca21cf8c8\") " pod="openshift-dns/node-resolver-hsg2d" Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710990 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/658cefc3-6811-4b18-b3a2-3af9ac357158-dbus\") pod \"global-pull-secret-syncer-5nf29\" (UID: \"658cefc3-6811-4b18-b3a2-3af9ac357158\") " pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711026 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-multus-conf-dir\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711026 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79786501-041d-4306-a959-b11ecc7e71d6-multus-daemon-config\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711051 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86rpn\" (UniqueName: \"kubernetes.io/projected/dfec4ce5-3bf7-4600-b522-9daca21cf8c8-kube-api-access-86rpn\") pod \"node-resolver-hsg2d\" (UID: \"dfec4ce5-3bf7-4600-b522-9daca21cf8c8\") " pod="openshift-dns/node-resolver-hsg2d" Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711070 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-cnibin\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711083 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-var-lib-cni-bin\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.711229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711086 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nkl6\" (UniqueName: \"kubernetes.io/projected/84115d28-35c1-4c55-9d01-c0f85ccfda09-kube-api-access-6nkl6\") pod \"node-ca-fx768\" (UID: \"84115d28-35c1-4c55-9d01-c0f85ccfda09\") " pod="openshift-image-registry/node-ca-fx768" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711108 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-var-lib-cni-multus\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711115 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-system-cni-dir\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.710986 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-hostroot\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711125 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/658cefc3-6811-4b18-b3a2-3af9ac357158-kubelet-config\") pod \"global-pull-secret-syncer-5nf29\" (UID: \"658cefc3-6811-4b18-b3a2-3af9ac357158\") " pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711181 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-run-multus-certs\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711390 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-os-release\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711414 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-run-k8s-cni-cncf-io\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711427 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-host-run-netns\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711462 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79786501-041d-4306-a959-b11ecc7e71d6-cni-binary-copy\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84115d28-35c1-4c55-9d01-c0f85ccfda09-serviceca\") pod \"node-ca-fx768\" (UID: \"84115d28-35c1-4c55-9d01-c0f85ccfda09\") " pod="openshift-image-registry/node-ca-fx768" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711506 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-system-cni-dir\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711520 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-multus-socket-dir-parent\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711539 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-etc-kubernetes\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711559 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-cnibin\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711561 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84115d28-35c1-4c55-9d01-c0f85ccfda09-host\") pod \"node-ca-fx768\" (UID: \"84115d28-35c1-4c55-9d01-c0f85ccfda09\") " pod="openshift-image-registry/node-ca-fx768" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711452 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79786501-041d-4306-a959-b11ecc7e71d6-multus-daemon-config\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711566 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-multus-socket-dir-parent\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.712120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711588 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79786501-041d-4306-a959-b11ecc7e71d6-etc-kubernetes\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.712669 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711595 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/658cefc3-6811-4b18-b3a2-3af9ac357158-dbus\") pod \"global-pull-secret-syncer-5nf29\" (UID: \"658cefc3-6811-4b18-b3a2-3af9ac357158\") " pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:26.712669 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.711612 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84115d28-35c1-4c55-9d01-c0f85ccfda09-host\") pod \"node-ca-fx768\" (UID: \"84115d28-35c1-4c55-9d01-c0f85ccfda09\") " pod="openshift-image-registry/node-ca-fx768" Jul 07 17:27:26.712669 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.712001 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84115d28-35c1-4c55-9d01-c0f85ccfda09-serviceca\") pod \"node-ca-fx768\" (UID: \"84115d28-35c1-4c55-9d01-c0f85ccfda09\") " pod="openshift-image-registry/node-ca-fx768" Jul 07 17:27:26.723582 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.723550 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nkl6\" (UniqueName: \"kubernetes.io/projected/84115d28-35c1-4c55-9d01-c0f85ccfda09-kube-api-access-6nkl6\") pod \"node-ca-fx768\" (UID: \"84115d28-35c1-4c55-9d01-c0f85ccfda09\") " pod="openshift-image-registry/node-ca-fx768" Jul 07 17:27:26.723790 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.723770 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86rpn\" (UniqueName: \"kubernetes.io/projected/dfec4ce5-3bf7-4600-b522-9daca21cf8c8-kube-api-access-86rpn\") pod \"node-resolver-hsg2d\" (UID: \"dfec4ce5-3bf7-4600-b522-9daca21cf8c8\") " pod="openshift-dns/node-resolver-hsg2d" Jul 07 17:27:26.723889 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.723863 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtzkl\" (UniqueName: \"kubernetes.io/projected/79786501-041d-4306-a959-b11ecc7e71d6-kube-api-access-dtzkl\") pod \"multus-bpm6b\" (UID: \"79786501-041d-4306-a959-b11ecc7e71d6\") " pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.804878 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.804790 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sxhbx" Jul 07 17:27:26.812789 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.812758 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:26.821639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.821610 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" Jul 07 17:27:26.828555 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.828530 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" Jul 07 17:27:26.836340 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.836129 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-77mtq" Jul 07 17:27:26.844745 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.844717 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pc8dw" Jul 07 17:27:26.852466 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.852441 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fx768" Jul 07 17:27:26.859946 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.859925 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bpm6b" Jul 07 17:27:26.866639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:26.866617 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hsg2d" Jul 07 17:27:27.115027 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:27.114926 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs\") pod \"network-metrics-daemon-7mtf5\" (UID: \"3e184935-aec2-42e9-8944-5199f4d6c1b9\") " pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:27.115194 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:27.115082 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:27:27.115194 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:27.115150 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs podName:3e184935-aec2-42e9-8944-5199f4d6c1b9 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:28.115131299 +0000 UTC m=+3.996767824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs") pod "network-metrics-daemon-7mtf5" (UID: "3e184935-aec2-42e9-8944-5199f4d6c1b9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:27:27.215585 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:27.215549 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd6b6\" (UniqueName: \"kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6\") pod \"network-check-target-47448\" (UID: \"688bff8d-e98a-4a83-a526-18eab4e56743\") " pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:27.215766 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:27.215646 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret\") pod \"global-pull-secret-syncer-5nf29\" (UID: \"658cefc3-6811-4b18-b3a2-3af9ac357158\") " pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:27.215766 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:27.215716 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:27:27.215766 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:27.215746 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:27:27.215766 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:27.215749 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:27:27.215766 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:27.215761 2562 projected.go:196] Error preparing data for projected volume kube-api-access-sd6b6 for pod openshift-network-diagnostics/network-check-target-47448: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:27:27.216034 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:27.215814 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret podName:658cefc3-6811-4b18-b3a2-3af9ac357158 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:28.215796567 +0000 UTC m=+4.097433090 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret") pod "global-pull-secret-syncer-5nf29" (UID: "658cefc3-6811-4b18-b3a2-3af9ac357158") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:27:27.216034 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:27.215834 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6 podName:688bff8d-e98a-4a83-a526-18eab4e56743 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:28.215825117 +0000 UTC m=+4.097461625 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sd6b6" (UniqueName: "kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6") pod "network-check-target-47448" (UID: "688bff8d-e98a-4a83-a526-18eab4e56743") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:27:27.561874 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:27.561780 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 17:22:26 +0000 UTC" deadline="2028-01-10 21:35:38.857447646 +0000 UTC" Jul 07 17:27:27.561874 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:27.561815 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13252h8m11.295636124s" Jul 07 17:27:27.632199 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:27.632175 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:27.632379 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:27.632220 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:27.632379 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:27.632303 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:27.632502 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:27.632408 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:27.637613 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:27.637547 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfec4ce5_3bf7_4600_b522_9daca21cf8c8.slice/crio-63aa0bee81096ad6147e30ce95d437a95c35d159b00d2ed1e85dd17ec32d9ac5 WatchSource:0}: Error finding container 63aa0bee81096ad6147e30ce95d437a95c35d159b00d2ed1e85dd17ec32d9ac5: Status 404 returned error can't find the container with id 63aa0bee81096ad6147e30ce95d437a95c35d159b00d2ed1e85dd17ec32d9ac5 Jul 07 17:27:27.639315 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:27.639292 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3f8bf7_5e2e_45b4_8b3d_56b50ebc873c.slice/crio-702be4bb00b67fb23b9f4198d22dcf48b2192b9960861c722577b3dd48a7e078 WatchSource:0}: Error finding container 702be4bb00b67fb23b9f4198d22dcf48b2192b9960861c722577b3dd48a7e078: Status 404 returned error can't find the container with id 702be4bb00b67fb23b9f4198d22dcf48b2192b9960861c722577b3dd48a7e078 Jul 07 17:27:27.640898 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:27:27.640871 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79786501_041d_4306_a959_b11ecc7e71d6.slice/crio-88720b36b7893695984125fd1ddd5a90d914360f6e29dcb76ec6705bd7198b23 WatchSource:0}: Error finding container 88720b36b7893695984125fd1ddd5a90d914360f6e29dcb76ec6705bd7198b23: Status 404 returned error can't find the container with id 88720b36b7893695984125fd1ddd5a90d914360f6e29dcb76ec6705bd7198b23 Jul 07 17:27:27.643636 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:27.643606 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sxhbx" event={"ID":"1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c","Type":"ContainerStarted","Data":"702be4bb00b67fb23b9f4198d22dcf48b2192b9960861c722577b3dd48a7e078"} Jul 07 17:27:27.656092 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:27.655462 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hsg2d" event={"ID":"dfec4ce5-3bf7-4600-b522-9daca21cf8c8","Type":"ContainerStarted","Data":"63aa0bee81096ad6147e30ce95d437a95c35d159b00d2ed1e85dd17ec32d9ac5"} Jul 07 17:27:28.121223 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.120893 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs\") pod \"network-metrics-daemon-7mtf5\" (UID: \"3e184935-aec2-42e9-8944-5199f4d6c1b9\") " pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:28.121385 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:28.121083 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:27:28.121385 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:28.121319 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs podName:3e184935-aec2-42e9-8944-5199f4d6c1b9 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:30.121298984 +0000 UTC m=+6.002935493 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs") pod "network-metrics-daemon-7mtf5" (UID: "3e184935-aec2-42e9-8944-5199f4d6c1b9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:27:28.221929 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.221890 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd6b6\" (UniqueName: \"kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6\") pod \"network-check-target-47448\" (UID: \"688bff8d-e98a-4a83-a526-18eab4e56743\") " pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:28.222137 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.221996 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret\") pod \"global-pull-secret-syncer-5nf29\" (UID: \"658cefc3-6811-4b18-b3a2-3af9ac357158\") " pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:28.222137 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:28.222114 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:27:28.222245 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:28.222174 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret podName:658cefc3-6811-4b18-b3a2-3af9ac357158 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:30.222156044 +0000 UTC m=+6.103792568 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret") pod "global-pull-secret-syncer-5nf29" (UID: "658cefc3-6811-4b18-b3a2-3af9ac357158") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:27:28.222549 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:28.222528 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:27:28.222549 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:28.222552 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:27:28.222692 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:28.222564 2562 projected.go:196] Error preparing data for projected volume kube-api-access-sd6b6 for pod openshift-network-diagnostics/network-check-target-47448: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:27:28.222692 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:28.222608 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6 podName:688bff8d-e98a-4a83-a526-18eab4e56743 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:30.222593798 +0000 UTC m=+6.104230304 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sd6b6" (UniqueName: "kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6") pod "network-check-target-47448" (UID: "688bff8d-e98a-4a83-a526-18eab4e56743") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:27:28.562902 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.562788 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 17:22:26 +0000 UTC" deadline="2028-02-26 19:10:19.666593598 +0000 UTC" Jul 07 17:27:28.562902 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.562852 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14377h42m51.10374477s" Jul 07 17:27:28.634242 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.633770 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:28.634242 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:28.633897 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:28.673671 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.673632 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-75.ec2.internal" event={"ID":"5e37f1de7e940fc5cfbd6ca0135ad2aa","Type":"ContainerStarted","Data":"40138d4568118ca745b53141fb444fbe03f360c5bf1f3679721b757612208c1a"} Jul 07 17:27:28.686812 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.686771 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" event={"ID":"3d96c74b-d6d5-488d-94f7-ad264ce0c44f","Type":"ContainerStarted","Data":"4ab5ec9ad65305b96ebb4ca71cd4549c36bf13678cb6bf8482831725df4f8bb7"} Jul 07 17:27:28.692116 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.692052 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-75.ec2.internal" podStartSLOduration=3.692031103 podStartE2EDuration="3.692031103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:27:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:27:28.691911517 +0000 UTC m=+4.573548045" watchObservedRunningTime="2026-07-07 17:27:28.692031103 +0000 UTC m=+4.573667611" Jul 07 17:27:28.702716 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.702675 2562 generic.go:356] "Generic (PLEG): container finished" podID="61717e98e5e11efe1087e5f0939585b1" containerID="b9e986c0acb062c73683a9ba4208e763c1d775bbf530e049afe8d0d639beff16" exitCode=0 Jul 07 17:27:28.702893 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.702801 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal" event={"ID":"61717e98e5e11efe1087e5f0939585b1","Type":"ContainerDied","Data":"b9e986c0acb062c73683a9ba4208e763c1d775bbf530e049afe8d0d639beff16"} Jul 07 17:27:28.716219 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.716091 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fx768" event={"ID":"84115d28-35c1-4c55-9d01-c0f85ccfda09","Type":"ContainerStarted","Data":"06178e2f06a6849c94bb5bef70b500356c9bbde670c488f23bbba689c262f6e5"} Jul 07 17:27:28.719595 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.719501 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pc8dw" event={"ID":"f81d1bd5-43a4-476a-b1c8-2a637c464bc9","Type":"ContainerStarted","Data":"0ea3e7d606a1fc62f188c5c1af658b7b951b6013418cece2f050018ba45ef72c"} Jul 07 17:27:28.724384 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.724347 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" event={"ID":"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d","Type":"ContainerStarted","Data":"4ad44392ec2bb15f758afe297dd2761e5db63a729461d3ed8d1b109c4248de85"} Jul 07 17:27:28.729665 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.729563 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" event={"ID":"7de1b8d9-18e1-4c4d-9202-fa628f48881d","Type":"ContainerStarted","Data":"c62b954a7e8745e6a50e69e5b13100295b87d36913650c5ba0f9eada07308305"} Jul 07 17:27:28.734412 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.734383 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-77mtq" event={"ID":"55f38dc4-f679-4b32-8f4e-6c3313461f0c","Type":"ContainerStarted","Data":"2fd1b2b06f7bc655496268b1271aa7f6b47453dccc3d0c8e29806f729686e491"} Jul 07 17:27:28.739872 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:28.739722 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bpm6b" event={"ID":"79786501-041d-4306-a959-b11ecc7e71d6","Type":"ContainerStarted","Data":"88720b36b7893695984125fd1ddd5a90d914360f6e29dcb76ec6705bd7198b23"} Jul 07 17:27:29.632322 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:29.632286 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:29.632698 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:29.632443 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:29.632866 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:29.632846 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:29.632987 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:29.632952 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:29.757215 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:29.757129 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal" event={"ID":"61717e98e5e11efe1087e5f0939585b1","Type":"ContainerStarted","Data":"d5306cc23d5e751e5f155d9f7d55b1f5fbc048536c116773ea11e2b1f03922eb"} Jul 07 17:27:30.144581 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:30.144544 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs\") pod \"network-metrics-daemon-7mtf5\" (UID: \"3e184935-aec2-42e9-8944-5199f4d6c1b9\") " pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:30.144791 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:30.144766 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:27:30.144851 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:30.144846 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs podName:3e184935-aec2-42e9-8944-5199f4d6c1b9 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:34.144825102 +0000 UTC m=+10.026461624 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs") pod "network-metrics-daemon-7mtf5" (UID: "3e184935-aec2-42e9-8944-5199f4d6c1b9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:27:30.245589 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:30.245525 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret\") pod \"global-pull-secret-syncer-5nf29\" (UID: \"658cefc3-6811-4b18-b3a2-3af9ac357158\") " pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:30.245589 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:30.245591 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd6b6\" (UniqueName: \"kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6\") pod \"network-check-target-47448\" (UID: \"688bff8d-e98a-4a83-a526-18eab4e56743\") " pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:30.245828 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:30.245797 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:27:30.245828 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:30.245821 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:27:30.245927 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:30.245835 2562 projected.go:196] Error preparing data for projected volume kube-api-access-sd6b6 for pod openshift-network-diagnostics/network-check-target-47448: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:27:30.245927 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:30.245899 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6 podName:688bff8d-e98a-4a83-a526-18eab4e56743 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:34.245882408 +0000 UTC m=+10.127518916 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sd6b6" (UniqueName: "kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6") pod "network-check-target-47448" (UID: "688bff8d-e98a-4a83-a526-18eab4e56743") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:27:30.246384 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:30.246357 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:27:30.246444 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:30.246414 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret podName:658cefc3-6811-4b18-b3a2-3af9ac357158 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:34.246399654 +0000 UTC m=+10.128036172 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret") pod "global-pull-secret-syncer-5nf29" (UID: "658cefc3-6811-4b18-b3a2-3af9ac357158") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:27:30.632059 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:30.631955 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:30.632218 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:30.632123 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:31.631892 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:31.631848 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:31.632390 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:31.631904 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:31.632390 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:31.632024 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:31.632583 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:31.632556 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:32.635089 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:32.634645 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:32.635089 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:32.634746 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:33.632206 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:33.632116 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:33.632377 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:33.632285 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:33.632674 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:33.632656 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:33.632773 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:33.632755 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:34.182207 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:34.182166 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs\") pod \"network-metrics-daemon-7mtf5\" (UID: \"3e184935-aec2-42e9-8944-5199f4d6c1b9\") " pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:34.182688 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:34.182316 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:27:34.182688 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:34.182384 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs podName:3e184935-aec2-42e9-8944-5199f4d6c1b9 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:42.182367042 +0000 UTC m=+18.064003551 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs") pod "network-metrics-daemon-7mtf5" (UID: "3e184935-aec2-42e9-8944-5199f4d6c1b9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:27:34.283016 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:34.282959 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd6b6\" (UniqueName: \"kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6\") pod \"network-check-target-47448\" (UID: \"688bff8d-e98a-4a83-a526-18eab4e56743\") " pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:34.283185 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:34.283077 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret\") pod \"global-pull-secret-syncer-5nf29\" (UID: \"658cefc3-6811-4b18-b3a2-3af9ac357158\") " pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:34.283273 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:34.283184 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:27:34.283273 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:34.283189 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:27:34.283273 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:34.283217 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:27:34.283273 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:34.283239 2562 projected.go:196] Error preparing data for projected volume kube-api-access-sd6b6 for pod openshift-network-diagnostics/network-check-target-47448: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:27:34.283273 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:34.283257 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret podName:658cefc3-6811-4b18-b3a2-3af9ac357158 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:42.283227364 +0000 UTC m=+18.164863869 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret") pod "global-pull-secret-syncer-5nf29" (UID: "658cefc3-6811-4b18-b3a2-3af9ac357158") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:27:34.283517 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:34.283296 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6 podName:688bff8d-e98a-4a83-a526-18eab4e56743 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:42.283277254 +0000 UTC m=+18.164913774 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sd6b6" (UniqueName: "kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6") pod "network-check-target-47448" (UID: "688bff8d-e98a-4a83-a526-18eab4e56743") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:27:34.632593 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:34.632515 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:34.632757 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:34.632624 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:35.632520 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:35.632480 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:35.632950 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:35.632604 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:35.632950 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:35.632668 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:35.632950 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:35.632802 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:36.634155 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:36.634115 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:36.634557 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:36.634243 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:37.631945 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:37.631903 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:37.632141 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:37.632074 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:37.632220 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:37.632145 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:37.632279 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:37.632265 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:38.632571 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:38.632480 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:38.633036 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:38.632622 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:39.632311 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:39.632270 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:39.632515 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:39.632411 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:39.632683 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:39.632276 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:39.633088 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:39.632764 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:40.632256 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:40.632211 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:40.632450 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:40.632365 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:41.631793 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:41.631750 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:41.632333 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:41.631763 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:41.632333 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:41.631920 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:41.632333 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:41.632013 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:42.242813 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:42.242778 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs\") pod \"network-metrics-daemon-7mtf5\" (UID: \"3e184935-aec2-42e9-8944-5199f4d6c1b9\") " pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:42.243022 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:42.242933 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:27:42.243022 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:42.243014 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs podName:3e184935-aec2-42e9-8944-5199f4d6c1b9 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:58.242997945 +0000 UTC m=+34.124634449 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs") pod "network-metrics-daemon-7mtf5" (UID: "3e184935-aec2-42e9-8944-5199f4d6c1b9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:27:42.343931 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:42.343891 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret\") pod \"global-pull-secret-syncer-5nf29\" (UID: \"658cefc3-6811-4b18-b3a2-3af9ac357158\") " pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:42.343931 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:42.343945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd6b6\" (UniqueName: \"kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6\") pod \"network-check-target-47448\" (UID: \"688bff8d-e98a-4a83-a526-18eab4e56743\") " pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:42.344196 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:42.344062 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:27:42.344196 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:42.344101 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:27:42.344196 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:42.344121 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:27:42.344196 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:42.344129 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret podName:658cefc3-6811-4b18-b3a2-3af9ac357158 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:58.344113636 +0000 UTC m=+34.225750141 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret") pod "global-pull-secret-syncer-5nf29" (UID: "658cefc3-6811-4b18-b3a2-3af9ac357158") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:27:42.344196 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:42.344133 2562 projected.go:196] Error preparing data for projected volume kube-api-access-sd6b6 for pod openshift-network-diagnostics/network-check-target-47448: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:27:42.344196 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:42.344182 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6 podName:688bff8d-e98a-4a83-a526-18eab4e56743 nodeName:}" failed. No retries permitted until 2026-07-07 17:27:58.344166246 +0000 UTC m=+34.225802754 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sd6b6" (UniqueName: "kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6") pod "network-check-target-47448" (UID: "688bff8d-e98a-4a83-a526-18eab4e56743") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:27:42.632527 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:42.632440 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:42.632940 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:42.632572 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:43.631931 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:43.631893 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:43.632145 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:43.631941 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:43.632145 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:43.632042 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:43.632258 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:43.632163 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:44.634533 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:44.634503 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:44.634946 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:44.634591 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:45.632026 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:45.631993 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:45.632195 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:45.632009 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:45.632195 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:45.632127 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:45.632324 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:45.632228 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:46.631704 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:46.631665 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:46.632077 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:46.631814 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:46.799790 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:46.799425 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" event={"ID":"7de1b8d9-18e1-4c4d-9202-fa628f48881d","Type":"ContainerStarted","Data":"2c8a12cf305bfdd17b1c1aa1de406db13d4cb4b2effa3b725973ab2b88e45399"} Jul 07 17:27:46.802177 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:46.801537 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-77mtq" event={"ID":"55f38dc4-f679-4b32-8f4e-6c3313461f0c","Type":"ContainerStarted","Data":"17d59e0a28d8222f3fd3e0b1cd6049ef0c0eacf5f991b48d6eab498f1f467a6d"} Jul 07 17:27:46.804412 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:46.804351 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bpm6b" event={"ID":"79786501-041d-4306-a959-b11ecc7e71d6","Type":"ContainerStarted","Data":"508dd50436175ad77d8f94fabe6eb1d4ac21eb73e04022f3bd38d13758f0215a"} Jul 07 17:27:46.810284 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:46.810219 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" event={"ID":"3d96c74b-d6d5-488d-94f7-ad264ce0c44f","Type":"ContainerStarted","Data":"c0d4caac7579d8954ff2348f8027c4904727d546ca627ae4f109585ed478e537"} Jul 07 17:27:46.821434 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:46.821379 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-75.ec2.internal" podStartSLOduration=21.821356501 podStartE2EDuration="21.821356501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:27:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:27:29.777358368 +0000 UTC m=+5.658994896" watchObservedRunningTime="2026-07-07 17:27:46.821356501 +0000 UTC m=+22.702993029" Jul 07 17:27:46.821757 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:46.821725 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-77mtq" podStartSLOduration=8.314925861 podStartE2EDuration="22.82171579s" totalImagesPullingTime="14.506789929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:27:46.821516632 +0000 UTC m=+22.703153153" watchObservedRunningTime="2026-07-07 17:27:46.82171579 +0000 UTC m=+22.703352318" Jul 07 17:27:46.866576 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:46.866523 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bpm6b" podStartSLOduration=2.854377382 podStartE2EDuration="21.866496638s" totalImagesPullingTime="19.012119256s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:27:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:27:46.845249314 +0000 UTC m=+22.726885842" watchObservedRunningTime="2026-07-07 17:27:46.866496638 +0000 UTC m=+22.748133165" Jul 07 17:27:46.867367 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:46.867329 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zs8ks" podStartSLOduration=3.9135189219999997 podStartE2EDuration="22.867315103s" totalImagesPullingTime="18.953796181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:27:46.866612019 +0000 UTC m=+22.748248549" watchObservedRunningTime="2026-07-07 17:27:46.867315103 +0000 UTC m=+22.748951632" Jul 07 17:27:47.632296 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:47.632046 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:47.632907 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:47.632046 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:47.632907 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:47.632355 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:47.632907 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:47.632400 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:47.814193 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:47.814155 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hsg2d" event={"ID":"dfec4ce5-3bf7-4600-b522-9daca21cf8c8","Type":"ContainerStarted","Data":"c88e3103654dec9c380a46e8e6f6d0d0e8fe0329a28fdb5e47e4b7f4c550b7ee"} Jul 07 17:27:47.815640 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:47.815610 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fx768" event={"ID":"84115d28-35c1-4c55-9d01-c0f85ccfda09","Type":"ContainerStarted","Data":"4814e2fb397ba9bf79549a3151868f1539bd04ecec0ccf7096cc0cb04330f205"} Jul 07 17:27:47.817195 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:47.817174 2562 generic.go:356] "Generic (PLEG): container finished" podID="f81d1bd5-43a4-476a-b1c8-2a637c464bc9" containerID="24df98f55debd25cda4380ec615c6494205950b0492723e6a461faf6202d136d" exitCode=0 Jul 07 17:27:47.817290 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:47.817232 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pc8dw" event={"ID":"f81d1bd5-43a4-476a-b1c8-2a637c464bc9","Type":"ContainerDied","Data":"24df98f55debd25cda4380ec615c6494205950b0492723e6a461faf6202d136d"} Jul 07 17:27:47.818756 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:47.818733 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" event={"ID":"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d","Type":"ContainerStarted","Data":"68bd33f098f097f4a493e268e1d172b4c242f19254e16f656c47ab31f56516fa"} Jul 07 17:27:47.821938 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:47.821918 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" event={"ID":"7de1b8d9-18e1-4c4d-9202-fa628f48881d","Type":"ContainerStarted","Data":"612f502082e158071d2aaacd3a8e18785d740bce5c3452865d476e6f67655fa7"} Jul 07 17:27:47.822056 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:47.821994 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" event={"ID":"7de1b8d9-18e1-4c4d-9202-fa628f48881d","Type":"ContainerStarted","Data":"f63ea247f3264b384334867ce5c9c478a30d62f5007ba3fe34a72ffb0957a8b1"} Jul 07 17:27:47.822056 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:47.822034 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" event={"ID":"7de1b8d9-18e1-4c4d-9202-fa628f48881d","Type":"ContainerStarted","Data":"598f531c961feb3db1dcbd1c71b1b644ebfee3e97dc4dc955072f25692a8dae6"} Jul 07 17:27:47.822166 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:47.822063 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" event={"ID":"7de1b8d9-18e1-4c4d-9202-fa628f48881d","Type":"ContainerStarted","Data":"19c0edeb596d1717cc5d794a0f45e9f57642153427df0621fc97e6efad7a1f39"} Jul 07 17:27:47.822166 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:47.822088 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" event={"ID":"7de1b8d9-18e1-4c4d-9202-fa628f48881d","Type":"ContainerStarted","Data":"b9a344f2aa4d1443abb1454e5bb826c6b53467b073784672cbd73705ddd826ff"} Jul 07 17:27:47.834915 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:47.834866 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hsg2d" podStartSLOduration=8.320470456 podStartE2EDuration="22.834852085s" totalImagesPullingTime="14.514381629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:27:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:27:47.834393973 +0000 UTC m=+23.716030501" watchObservedRunningTime="2026-07-07 17:27:47.834852085 +0000 UTC m=+23.716488613" Jul 07 17:27:47.878777 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:47.878723 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fx768" podStartSLOduration=3.924162748 podStartE2EDuration="22.878706392s" totalImagesPullingTime="18.954543644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:27:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:27:47.878152956 +0000 UTC m=+23.759789482" watchObservedRunningTime="2026-07-07 17:27:47.878706392 +0000 UTC m=+23.760342920" Jul 07 17:27:48.249205 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:48.249160 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-77mtq" Jul 07 17:27:48.249862 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:48.249832 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-77mtq" Jul 07 17:27:48.632414 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:48.632384 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:48.632898 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:48.632511 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:48.826058 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:48.826021 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sxhbx" event={"ID":"1e3f8bf7-5e2e-45b4-8b3d-56b50ebc873c","Type":"ContainerStarted","Data":"79f7f0aa2f0dc7795bafd7a51ae88e1a41545d91b5bc8266d1f7fdc036d2d1aa"} Jul 07 17:27:48.826474 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:48.826445 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-77mtq" Jul 07 17:27:48.827137 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:48.827117 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-77mtq" Jul 07 17:27:48.845666 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:48.845610 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sxhbx" podStartSLOduration=5.883772372 podStartE2EDuration="24.845594323s" totalImagesPullingTime="18.961821951s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:27:48.845407642 +0000 UTC m=+24.727044170" watchObservedRunningTime="2026-07-07 17:27:48.845594323 +0000 UTC m=+24.727230850" Jul 07 17:27:48.883770 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:48.883693 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 17:27:49.567519 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:49.567395 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T17:27:48.883721203Z","UUID":"cf076260-18ee-401b-a1ad-7fab038e778a","Handler":null,"Name":"","Endpoint":""} Jul 07 17:27:49.569399 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:49.569376 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 17:27:49.569534 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:49.569424 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 17:27:49.632127 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:49.632039 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:49.632295 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:49.632181 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:49.632546 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:49.632530 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:49.632851 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:49.632613 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:49.831946 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:49.831901 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" event={"ID":"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d","Type":"ContainerStarted","Data":"f319bdb4ac5b8ac22fa004567faf337a8f614ecce10f4901384953258f3e04df"} Jul 07 17:27:49.836301 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:49.836253 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" event={"ID":"7de1b8d9-18e1-4c4d-9202-fa628f48881d","Type":"ContainerStarted","Data":"903029e5e39b5a1914fb566f66e909c3ff34909cd6a65a977d8182024b649116"} Jul 07 17:27:50.632739 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:50.632475 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:50.633215 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:50.632796 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:50.840490 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:50.840439 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" event={"ID":"bfe12642-5f4d-478a-8cf5-1c8928ad0f4d","Type":"ContainerStarted","Data":"3c8ff7c18c40a687151182836277aa9450adb794d5ef8c324ebc617c339d5d36"} Jul 07 17:27:50.868762 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:50.868692 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fvg5v" podStartSLOduration=4.762194591 podStartE2EDuration="26.868677575s" totalImagesPullingTime="22.106482984s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:27:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:27:50.866752622 +0000 UTC m=+26.748389150" watchObservedRunningTime="2026-07-07 17:27:50.868677575 +0000 UTC m=+26.750314136" Jul 07 17:27:51.631515 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:51.631489 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:51.631671 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:51.631487 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:51.631671 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:51.631607 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:51.631776 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:51.631702 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:51.848909 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:51.848818 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" event={"ID":"7de1b8d9-18e1-4c4d-9202-fa628f48881d","Type":"ContainerStarted","Data":"23a0da3c7116182f03b6de1df9dda6b7cacd0935fe73b4c809a2089879bd032f"} Jul 07 17:27:51.849750 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:51.849031 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:51.849750 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:51.849087 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:51.849750 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:51.849122 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:51.869038 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:51.869007 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:51.869800 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:51.869778 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:27:51.893277 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:51.893218 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" podStartSLOduration=8.866204191 podStartE2EDuration="27.893202055s" totalImagesPullingTime="19.026997864s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:27:51.891743966 +0000 UTC m=+27.773380493" watchObservedRunningTime="2026-07-07 17:27:51.893202055 +0000 UTC m=+27.774838583" Jul 07 17:27:52.632606 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:52.632562 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:52.632779 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:52.632694 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:53.631980 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:53.631721 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:53.632388 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:53.632100 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:53.632483 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:53.631723 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:53.632619 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:53.632592 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:54.048882 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:54.048790 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5nf29"] Jul 07 17:27:54.049116 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:54.048902 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:54.049116 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:54.049015 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:54.050771 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:54.050744 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-47448"] Jul 07 17:27:54.050865 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:54.050831 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:54.050933 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:54.050905 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:54.053544 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:54.053521 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7mtf5"] Jul 07 17:27:54.053652 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:54.053620 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:54.053764 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:54.053735 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:54.857769 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:54.857736 2562 generic.go:356] "Generic (PLEG): container finished" podID="f81d1bd5-43a4-476a-b1c8-2a637c464bc9" containerID="bd3cd95f41e04847c566ee1e29c0ae07ac29764e44ddb75e86502fa5f7e26481" exitCode=0 Jul 07 17:27:54.858198 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:54.857779 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pc8dw" event={"ID":"f81d1bd5-43a4-476a-b1c8-2a637c464bc9","Type":"ContainerDied","Data":"bd3cd95f41e04847c566ee1e29c0ae07ac29764e44ddb75e86502fa5f7e26481"} Jul 07 17:27:55.632503 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:55.632479 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:55.632629 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:55.632560 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:55.632687 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:55.632674 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:55.632740 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:55.632690 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:55.632809 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:55.632791 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:55.632907 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:55.632889 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:55.861631 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:55.861549 2562 generic.go:356] "Generic (PLEG): container finished" podID="f81d1bd5-43a4-476a-b1c8-2a637c464bc9" containerID="a266efc67a72e3b96a8bd52214356ca91160547f3e5a84c093a1f8260b4df613" exitCode=0 Jul 07 17:27:55.861631 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:55.861612 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pc8dw" event={"ID":"f81d1bd5-43a4-476a-b1c8-2a637c464bc9","Type":"ContainerDied","Data":"a266efc67a72e3b96a8bd52214356ca91160547f3e5a84c093a1f8260b4df613"} Jul 07 17:27:56.868736 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:56.866315 2562 generic.go:356] "Generic (PLEG): container finished" podID="f81d1bd5-43a4-476a-b1c8-2a637c464bc9" containerID="26a509fb31c7d4d28c17e82292cb60343ab1492a2cd7a6edf5664acf322aa92a" exitCode=0 Jul 07 17:27:56.868736 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:56.866615 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pc8dw" event={"ID":"f81d1bd5-43a4-476a-b1c8-2a637c464bc9","Type":"ContainerDied","Data":"26a509fb31c7d4d28c17e82292cb60343ab1492a2cd7a6edf5664acf322aa92a"} Jul 07 17:27:57.631644 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:57.631556 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:57.631644 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:57.631593 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:57.631948 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:57.631556 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:57.631948 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:57.631690 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtf5" podUID="3e184935-aec2-42e9-8944-5199f4d6c1b9" Jul 07 17:27:57.631948 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:57.631775 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47448" podUID="688bff8d-e98a-4a83-a526-18eab4e56743" Jul 07 17:27:57.631948 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:57.631827 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nf29" podUID="658cefc3-6811-4b18-b3a2-3af9ac357158" Jul 07 17:27:58.265486 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:58.265446 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs\") pod \"network-metrics-daemon-7mtf5\" (UID: \"3e184935-aec2-42e9-8944-5199f4d6c1b9\") " pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:58.265936 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:58.265614 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:27:58.265936 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:58.265682 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs podName:3e184935-aec2-42e9-8944-5199f4d6c1b9 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:30.265663979 +0000 UTC m=+66.147300488 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs") pod "network-metrics-daemon-7mtf5" (UID: "3e184935-aec2-42e9-8944-5199f4d6c1b9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:27:58.366671 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:58.366635 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret\") pod \"global-pull-secret-syncer-5nf29\" (UID: \"658cefc3-6811-4b18-b3a2-3af9ac357158\") " pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:58.366671 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:58.366685 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd6b6\" (UniqueName: \"kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6\") pod \"network-check-target-47448\" (UID: \"688bff8d-e98a-4a83-a526-18eab4e56743\") " pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:58.366939 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:58.366798 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:27:58.366939 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:58.366812 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:27:58.366939 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:58.366833 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:27:58.366939 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:58.366845 2562 projected.go:196] Error preparing data for projected volume kube-api-access-sd6b6 for pod openshift-network-diagnostics/network-check-target-47448: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:27:58.366939 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:58.366861 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret podName:658cefc3-6811-4b18-b3a2-3af9ac357158 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:30.366846474 +0000 UTC m=+66.248482979 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret") pod "global-pull-secret-syncer-5nf29" (UID: "658cefc3-6811-4b18-b3a2-3af9ac357158") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:27:58.366939 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:58.366891 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6 podName:688bff8d-e98a-4a83-a526-18eab4e56743 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:30.3668772 +0000 UTC m=+66.248513719 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sd6b6" (UniqueName: "kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6") pod "network-check-target-47448" (UID: "688bff8d-e98a-4a83-a526-18eab4e56743") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:27:59.374491 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.374465 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-75.ec2.internal" event="NodeReady" Jul 07 17:27:59.374847 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.374578 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 17:27:59.421615 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.421514 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7fc965b778-q5jc7"] Jul 07 17:27:59.424639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.424615 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7fbfb58cb7-68cb8"] Jul 07 17:27:59.424768 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.424747 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.427467 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.427441 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 17:27:59.427595 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.427561 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 17:27:59.427821 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.427799 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5"] Jul 07 17:27:59.427925 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.427911 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:27:59.428181 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.428161 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 17:27:59.428897 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.428606 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lzxpj\"" Jul 07 17:27:59.431927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.430768 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 17:27:59.431927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.431082 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 17:27:59.431927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.431329 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 17:27:59.431927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.431544 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 17:27:59.432182 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.432054 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 17:27:59.432725 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.432708 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xm8zg\"" Jul 07 17:27:59.432869 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.432828 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:27:59.432869 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.432736 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 17:27:59.433313 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.432746 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds5pp"] Jul 07 17:27:59.435750 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.435730 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 17:27:59.438118 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.437694 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 17:27:59.438118 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.437840 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv"] Jul 07 17:27:59.438118 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.437911 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 17:27:59.438118 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.437950 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 17:27:59.438118 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.438044 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-tg4q6\"" Jul 07 17:27:59.438380 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.438127 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 17:27:59.438380 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.438309 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds5pp" Jul 07 17:27:59.442578 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.441007 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9"] Jul 07 17:27:59.447194 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.443808 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-lzx9m\"" Jul 07 17:27:59.447194 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.443840 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:27:59.447194 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.444263 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv" Jul 07 17:27:59.447194 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.445362 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 17:27:59.448136 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.448110 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k"] Jul 07 17:27:59.448267 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.448212 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 07 17:27:59.448443 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.448427 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 07 17:27:59.448625 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.448611 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:27:59.448862 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.448845 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 07 17:27:59.449133 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.449114 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9" Jul 07 17:27:59.450598 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.450563 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-sq4v2\"" Jul 07 17:27:59.454012 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.453406 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-nxsw4"] Jul 07 17:27:59.454012 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.453541 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" Jul 07 17:27:59.454012 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.453836 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 17:27:59.454799 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.454349 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-n25cs\"" Jul 07 17:27:59.454799 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.454356 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 17:27:59.454799 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.454558 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 17:27:59.454799 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.454677 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:27:59.456722 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.456457 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 07 17:27:59.456817 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.456737 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jpt5n\"" Jul 07 17:27:59.456951 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.456937 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:27:59.458091 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.457160 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-5gff8"] Jul 07 17:27:59.458091 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.457240 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 07 17:27:59.458091 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.457401 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:27:59.460237 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.460220 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 17:27:59.460563 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.460541 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-x65k8\"" Jul 07 17:27:59.460716 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.460695 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 17:27:59.460857 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.460736 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:27:59.461043 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.460543 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 17:27:59.462000 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.461953 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds5pp"] Jul 07 17:27:59.462091 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.462020 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5"] Jul 07 17:27:59.462091 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.462041 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9"] Jul 07 17:27:59.462091 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.462061 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv"] Jul 07 17:27:59.462091 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.462081 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7fbfb58cb7-68cb8"] Jul 07 17:27:59.462278 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.462101 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-nxsw4"] Jul 07 17:27:59.462278 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.462123 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mwh52"] Jul 07 17:27:59.462278 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.462079 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.466424 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.466397 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 17:27:59.466960 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.466932 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fc965b778-q5jc7"] Jul 07 17:27:59.467072 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.467016 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-n4h5f"] Jul 07 17:27:59.467628 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.467610 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 17:27:59.467803 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.467783 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 17:27:59.468010 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.467626 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mwh52" Jul 07 17:27:59.469631 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.469607 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-thvx9\"" Jul 07 17:27:59.469985 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.469948 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 17:27:59.473906 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.472737 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 17:27:59.473906 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.472916 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tpq4p\"" Jul 07 17:27:59.473906 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.472744 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 17:27:59.473906 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.473317 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 17:27:59.475502 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.475360 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-5gff8"] Jul 07 17:27:59.475502 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.475398 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mwh52"] Jul 07 17:27:59.475502 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.475415 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k"] Jul 07 17:27:59.475502 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.475444 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n4h5f"] Jul 07 17:27:59.475740 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.475573 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n4h5f" Jul 07 17:27:59.475788 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.475776 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 17:27:59.478565 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.478546 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 17:27:59.478645 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.478614 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 17:27:59.479219 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.479190 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 17:27:59.479467 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.479451 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-78k7c\"" Jul 07 17:27:59.574954 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.574924 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qrzv\" (UniqueName: \"kubernetes.io/projected/b6d93f73-0d7b-44c8-bd96-b707875dd0af-kube-api-access-5qrzv\") pod \"volume-data-source-validator-5d456fdbcd-ds5pp\" (UID: \"b6d93f73-0d7b-44c8-bd96-b707875dd0af\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds5pp" Jul 07 17:27:59.574954 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.574955 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:27:59.575181 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.574996 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52xr5\" (UID: \"2308b91e-a103-451e-b06f-fc5bb0b50c97\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:27:59.575181 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575025 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77n5f\" (UniqueName: \"kubernetes.io/projected/c72df9fa-884f-4fac-8557-199256e8c242-kube-api-access-77n5f\") pod \"service-ca-operator-5d46d9ffd4-hrfnv\" (UID: \"c72df9fa-884f-4fac-8557-199256e8c242\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv" Jul 07 17:27:59.575181 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575048 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15b2d389-16ed-4641-bffb-91298e7516c2-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.575181 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575066 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4a2ea0f-ff58-4688-b947-46d4352afc62-installation-pull-secrets\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.575181 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575083 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nfjz\" (UniqueName: \"kubernetes.io/projected/2308b91e-a103-451e-b06f-fc5bb0b50c97-kube-api-access-6nfjz\") pod \"cluster-monitoring-operator-64786d9cbb-52xr5\" (UID: \"2308b91e-a103-451e-b06f-fc5bb0b50c97\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:27:59.575181 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575124 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cs4f\" (UniqueName: \"kubernetes.io/projected/9dcbb03e-a464-408b-a012-8fc6a386ef16-kube-api-access-6cs4f\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:27:59.575181 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575160 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-bound-sa-token\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.575181 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575179 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b398fe77-3362-4eaa-b811-95a3d049fcb5-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-mr9z9\" (UID: \"b398fe77-3362-4eaa-b811-95a3d049fcb5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9" Jul 07 17:27:59.575511 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575204 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4a2ea0f-ff58-4688-b947-46d4352afc62-trusted-ca\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.575511 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575252 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8eea6af9-b0e3-4bf5-9ee8-aa45037566a3-trusted-ca\") pod \"console-operator-68b757fb67-nxsw4\" (UID: \"8eea6af9-b0e3-4bf5-9ee8-aa45037566a3\") " pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:27:59.575511 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575290 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dcbb03e-a464-408b-a012-8fc6a386ef16-tmp-dir\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:27:59.575511 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575316 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:27:59.575511 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575344 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9dcbb03e-a464-408b-a012-8fc6a386ef16-config-volume\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:27:59.575511 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575367 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15b2d389-16ed-4641-bffb-91298e7516c2-tmp\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.575511 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575401 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l42k\" (UID: \"8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" Jul 07 17:27:59.575511 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575418 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gz85\" (UniqueName: \"kubernetes.io/projected/b398fe77-3362-4eaa-b811-95a3d049fcb5-kube-api-access-2gz85\") pod \"kube-storage-version-migrator-operator-846bf668f7-mr9z9\" (UID: \"b398fe77-3362-4eaa-b811-95a3d049fcb5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9" Jul 07 17:27:59.575511 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575436 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2vwt\" (UniqueName: \"kubernetes.io/projected/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-kube-api-access-z2vwt\") pod \"cluster-samples-operator-56fcf86bf-4l42k\" (UID: \"8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" Jul 07 17:27:59.575511 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575452 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/15b2d389-16ed-4641-bffb-91298e7516c2-snapshots\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.575851 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575525 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c72df9fa-884f-4fac-8557-199256e8c242-config\") pod \"service-ca-operator-5d46d9ffd4-hrfnv\" (UID: \"c72df9fa-884f-4fac-8557-199256e8c242\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv" Jul 07 17:27:59.575851 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575542 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/15b2d389-16ed-4641-bffb-91298e7516c2-serving-cert\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.575851 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575558 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t2mf\" (UniqueName: \"kubernetes.io/projected/74900825-b343-4085-aee1-d3e8c0bf2064-kube-api-access-5t2mf\") pod \"ingress-canary-mwh52\" (UID: \"74900825-b343-4085-aee1-d3e8c0bf2064\") " pod="openshift-ingress-canary/ingress-canary-mwh52" Jul 07 17:27:59.575851 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575593 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8eea6af9-b0e3-4bf5-9ee8-aa45037566a3-config\") pod \"console-operator-68b757fb67-nxsw4\" (UID: \"8eea6af9-b0e3-4bf5-9ee8-aa45037566a3\") " pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:27:59.575851 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575622 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h75qf\" (UniqueName: \"kubernetes.io/projected/8eea6af9-b0e3-4bf5-9ee8-aa45037566a3-kube-api-access-h75qf\") pod \"console-operator-68b757fb67-nxsw4\" (UID: \"8eea6af9-b0e3-4bf5-9ee8-aa45037566a3\") " pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:27:59.575851 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575656 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4a2ea0f-ff58-4688-b947-46d4352afc62-ca-trust-extracted\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.575851 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575676 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15b2d389-16ed-4641-bffb-91298e7516c2-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.575851 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575699 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4a2ea0f-ff58-4688-b947-46d4352afc62-image-registry-private-configuration\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.575851 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575748 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-stats-auth\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:27:59.575851 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575770 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c72df9fa-884f-4fac-8557-199256e8c242-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-hrfnv\" (UID: \"c72df9fa-884f-4fac-8557-199256e8c242\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv" Jul 07 17:27:59.575851 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575808 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74900825-b343-4085-aee1-d3e8c0bf2064-cert\") pod \"ingress-canary-mwh52\" (UID: \"74900825-b343-4085-aee1-d3e8c0bf2064\") " pod="openshift-ingress-canary/ingress-canary-mwh52" Jul 07 17:27:59.575851 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575829 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgkn8\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-kube-api-access-qgkn8\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.575851 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575847 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2308b91e-a103-451e-b06f-fc5bb0b50c97-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-52xr5\" (UID: \"2308b91e-a103-451e-b06f-fc5bb0b50c97\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:27:59.576283 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575861 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8eea6af9-b0e3-4bf5-9ee8-aa45037566a3-serving-cert\") pod \"console-operator-68b757fb67-nxsw4\" (UID: \"8eea6af9-b0e3-4bf5-9ee8-aa45037566a3\") " pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:27:59.576283 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575875 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-certificates\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.576283 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575899 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7kvh\" (UniqueName: \"kubernetes.io/projected/b6c8843e-708e-4e0d-a538-e35c93a4beec-kube-api-access-p7kvh\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:27:59.576283 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575922 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.576283 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kqnv\" (UniqueName: \"kubernetes.io/projected/15b2d389-16ed-4641-bffb-91298e7516c2-kube-api-access-6kqnv\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.576283 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.575952 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b398fe77-3362-4eaa-b811-95a3d049fcb5-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-mr9z9\" (UID: \"b398fe77-3362-4eaa-b811-95a3d049fcb5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9" Jul 07 17:27:59.576283 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.576011 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-default-certificate\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:27:59.576283 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.576027 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:27:59.631726 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.631696 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:27:59.631904 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.631699 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:27:59.631990 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.631705 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:27:59.634419 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.634395 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 17:27:59.634419 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.634415 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hxlp2\"" Jul 07 17:27:59.634582 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.634433 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 17:27:59.634582 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.634476 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 17:27:59.634582 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.634561 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 17:27:59.634935 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.634921 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v8q6k\"" Jul 07 17:27:59.677306 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677216 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-bound-sa-token\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.677306 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677253 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b398fe77-3362-4eaa-b811-95a3d049fcb5-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-mr9z9\" (UID: \"b398fe77-3362-4eaa-b811-95a3d049fcb5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9" Jul 07 17:27:59.677306 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677274 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4a2ea0f-ff58-4688-b947-46d4352afc62-trusted-ca\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.677306 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677291 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8eea6af9-b0e3-4bf5-9ee8-aa45037566a3-trusted-ca\") pod \"console-operator-68b757fb67-nxsw4\" (UID: \"8eea6af9-b0e3-4bf5-9ee8-aa45037566a3\") " pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:27:59.677639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dcbb03e-a464-408b-a012-8fc6a386ef16-tmp-dir\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:27:59.677639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677351 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:27:59.677639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677376 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9dcbb03e-a464-408b-a012-8fc6a386ef16-config-volume\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:27:59.677639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677398 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15b2d389-16ed-4641-bffb-91298e7516c2-tmp\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.677639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677425 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l42k\" (UID: \"8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" Jul 07 17:27:59.677639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gz85\" (UniqueName: \"kubernetes.io/projected/b398fe77-3362-4eaa-b811-95a3d049fcb5-kube-api-access-2gz85\") pod \"kube-storage-version-migrator-operator-846bf668f7-mr9z9\" (UID: \"b398fe77-3362-4eaa-b811-95a3d049fcb5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9" Jul 07 17:27:59.677639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677482 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2vwt\" (UniqueName: \"kubernetes.io/projected/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-kube-api-access-z2vwt\") pod \"cluster-samples-operator-56fcf86bf-4l42k\" (UID: \"8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" Jul 07 17:27:59.677639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677508 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/15b2d389-16ed-4641-bffb-91298e7516c2-snapshots\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.677639 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:59.677535 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle podName:b6c8843e-708e-4e0d-a538-e35c93a4beec nodeName:}" failed. No retries permitted until 2026-07-07 17:28:00.177509204 +0000 UTC m=+36.059145710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle") pod "router-default-7fbfb58cb7-68cb8" (UID: "b6c8843e-708e-4e0d-a538-e35c93a4beec") : configmap references non-existent config key: service-ca.crt Jul 07 17:27:59.677639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677631 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c72df9fa-884f-4fac-8557-199256e8c242-config\") pod \"service-ca-operator-5d46d9ffd4-hrfnv\" (UID: \"c72df9fa-884f-4fac-8557-199256e8c242\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv" Jul 07 17:27:59.678169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677663 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/15b2d389-16ed-4641-bffb-91298e7516c2-serving-cert\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.678169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677698 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t2mf\" (UniqueName: \"kubernetes.io/projected/74900825-b343-4085-aee1-d3e8c0bf2064-kube-api-access-5t2mf\") pod \"ingress-canary-mwh52\" (UID: \"74900825-b343-4085-aee1-d3e8c0bf2064\") " pod="openshift-ingress-canary/ingress-canary-mwh52" Jul 07 17:27:59.678169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677723 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8eea6af9-b0e3-4bf5-9ee8-aa45037566a3-config\") pod \"console-operator-68b757fb67-nxsw4\" (UID: \"8eea6af9-b0e3-4bf5-9ee8-aa45037566a3\") " pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:27:59.678169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677752 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h75qf\" (UniqueName: \"kubernetes.io/projected/8eea6af9-b0e3-4bf5-9ee8-aa45037566a3-kube-api-access-h75qf\") pod \"console-operator-68b757fb67-nxsw4\" (UID: \"8eea6af9-b0e3-4bf5-9ee8-aa45037566a3\") " pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:27:59.678169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677791 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4a2ea0f-ff58-4688-b947-46d4352afc62-ca-trust-extracted\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.678169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677821 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15b2d389-16ed-4641-bffb-91298e7516c2-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.678169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677858 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4a2ea0f-ff58-4688-b947-46d4352afc62-image-registry-private-configuration\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.678169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677903 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-stats-auth\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:27:59.678169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677936 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c72df9fa-884f-4fac-8557-199256e8c242-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-hrfnv\" (UID: \"c72df9fa-884f-4fac-8557-199256e8c242\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv" Jul 07 17:27:59.678169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.677955 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b398fe77-3362-4eaa-b811-95a3d049fcb5-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-mr9z9\" (UID: \"b398fe77-3362-4eaa-b811-95a3d049fcb5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9" Jul 07 17:27:59.678169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678005 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74900825-b343-4085-aee1-d3e8c0bf2064-cert\") pod \"ingress-canary-mwh52\" (UID: \"74900825-b343-4085-aee1-d3e8c0bf2064\") " pod="openshift-ingress-canary/ingress-canary-mwh52" Jul 07 17:27:59.678169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678006 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9dcbb03e-a464-408b-a012-8fc6a386ef16-config-volume\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:27:59.678169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678059 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgkn8\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-kube-api-access-qgkn8\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.678169 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:59.678087 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 17:27:59.678169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678099 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2308b91e-a103-451e-b06f-fc5bb0b50c97-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-52xr5\" (UID: \"2308b91e-a103-451e-b06f-fc5bb0b50c97\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:27:59.678169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678124 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8eea6af9-b0e3-4bf5-9ee8-aa45037566a3-serving-cert\") pod \"console-operator-68b757fb67-nxsw4\" (UID: \"8eea6af9-b0e3-4bf5-9ee8-aa45037566a3\") " pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:27:59.678905 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:59.678151 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls podName:8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b nodeName:}" failed. No retries permitted until 2026-07-07 17:28:00.178133089 +0000 UTC m=+36.059769608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-4l42k" (UID: "8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b") : secret "samples-operator-tls" not found Jul 07 17:27:59.678905 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678188 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-certificates\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.678905 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678223 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7kvh\" (UniqueName: \"kubernetes.io/projected/b6c8843e-708e-4e0d-a538-e35c93a4beec-kube-api-access-p7kvh\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:27:59.678905 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678198 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/15b2d389-16ed-4641-bffb-91298e7516c2-snapshots\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.678905 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678251 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.678905 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678277 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kqnv\" (UniqueName: \"kubernetes.io/projected/15b2d389-16ed-4641-bffb-91298e7516c2-kube-api-access-6kqnv\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.678905 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678301 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b398fe77-3362-4eaa-b811-95a3d049fcb5-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-mr9z9\" (UID: \"b398fe77-3362-4eaa-b811-95a3d049fcb5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9" Jul 07 17:27:59.678905 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678333 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-default-certificate\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:27:59.678905 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678356 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:27:59.678905 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678375 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4a2ea0f-ff58-4688-b947-46d4352afc62-trusted-ca\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.678905 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678385 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qrzv\" (UniqueName: \"kubernetes.io/projected/b6d93f73-0d7b-44c8-bd96-b707875dd0af-kube-api-access-5qrzv\") pod \"volume-data-source-validator-5d456fdbcd-ds5pp\" (UID: \"b6d93f73-0d7b-44c8-bd96-b707875dd0af\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds5pp" Jul 07 17:27:59.678905 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678415 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8eea6af9-b0e3-4bf5-9ee8-aa45037566a3-trusted-ca\") pod \"console-operator-68b757fb67-nxsw4\" (UID: \"8eea6af9-b0e3-4bf5-9ee8-aa45037566a3\") " pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:27:59.678905 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678431 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:27:59.678905 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678472 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52xr5\" (UID: \"2308b91e-a103-451e-b06f-fc5bb0b50c97\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:27:59.678905 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678497 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c72df9fa-884f-4fac-8557-199256e8c242-config\") pod \"service-ca-operator-5d46d9ffd4-hrfnv\" (UID: \"c72df9fa-884f-4fac-8557-199256e8c242\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv" Jul 07 17:27:59.679591 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678510 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77n5f\" (UniqueName: \"kubernetes.io/projected/c72df9fa-884f-4fac-8557-199256e8c242-kube-api-access-77n5f\") pod \"service-ca-operator-5d46d9ffd4-hrfnv\" (UID: \"c72df9fa-884f-4fac-8557-199256e8c242\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv" Jul 07 17:27:59.679591 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:59.678628 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 17:27:59.679591 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:59.678682 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs podName:b6c8843e-708e-4e0d-a538-e35c93a4beec nodeName:}" failed. No retries permitted until 2026-07-07 17:28:00.178667732 +0000 UTC m=+36.060304241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs") pod "router-default-7fbfb58cb7-68cb8" (UID: "b6c8843e-708e-4e0d-a538-e35c93a4beec") : secret "router-metrics-certs-default" not found Jul 07 17:27:59.679591 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:59.678698 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 17:27:59.679591 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:59.678742 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:27:59.679591 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:59.678748 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls podName:2308b91e-a103-451e-b06f-fc5bb0b50c97 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:00.178731722 +0000 UTC m=+36.060368230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-52xr5" (UID: "2308b91e-a103-451e-b06f-fc5bb0b50c97") : secret "cluster-monitoring-operator-tls" not found Jul 07 17:27:59.679591 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:59.678753 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fc965b778-q5jc7: secret "image-registry-tls" not found Jul 07 17:27:59.679591 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:59.678786 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls podName:e4a2ea0f-ff58-4688-b947-46d4352afc62 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:00.178775262 +0000 UTC m=+36.060411771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls") pod "image-registry-7fc965b778-q5jc7" (UID: "e4a2ea0f-ff58-4688-b947-46d4352afc62") : secret "image-registry-tls" not found Jul 07 17:27:59.679591 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.678935 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dcbb03e-a464-408b-a012-8fc6a386ef16-tmp-dir\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:27:59.679591 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.679242 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2308b91e-a103-451e-b06f-fc5bb0b50c97-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-52xr5\" (UID: \"2308b91e-a103-451e-b06f-fc5bb0b50c97\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:27:59.679591 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.679259 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-certificates\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.679591 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:59.679532 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:27:59.679591 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:27:59.679574 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls podName:9dcbb03e-a464-408b-a012-8fc6a386ef16 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:00.17956039 +0000 UTC m=+36.061196901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls") pod "dns-default-n4h5f" (UID: "9dcbb03e-a464-408b-a012-8fc6a386ef16") : secret "dns-default-metrics-tls" not found Jul 07 17:27:59.680171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.679618 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15b2d389-16ed-4641-bffb-91298e7516c2-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.680171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.679764 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15b2d389-16ed-4641-bffb-91298e7516c2-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.680171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.679807 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4a2ea0f-ff58-4688-b947-46d4352afc62-installation-pull-secrets\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.680171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.679840 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nfjz\" (UniqueName: \"kubernetes.io/projected/2308b91e-a103-451e-b06f-fc5bb0b50c97-kube-api-access-6nfjz\") pod \"cluster-monitoring-operator-64786d9cbb-52xr5\" (UID: \"2308b91e-a103-451e-b06f-fc5bb0b50c97\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:27:59.680171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.679874 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cs4f\" (UniqueName: \"kubernetes.io/projected/9dcbb03e-a464-408b-a012-8fc6a386ef16-kube-api-access-6cs4f\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:27:59.680419 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.680206 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8eea6af9-b0e3-4bf5-9ee8-aa45037566a3-config\") pod \"console-operator-68b757fb67-nxsw4\" (UID: \"8eea6af9-b0e3-4bf5-9ee8-aa45037566a3\") " pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:27:59.680853 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.680797 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15b2d389-16ed-4641-bffb-91298e7516c2-tmp\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.681609 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.681294 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4a2ea0f-ff58-4688-b947-46d4352afc62-ca-trust-extracted\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.682395 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.682373 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15b2d389-16ed-4641-bffb-91298e7516c2-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.683643 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.683599 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/15b2d389-16ed-4641-bffb-91298e7516c2-serving-cert\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.683730 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.683684 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-stats-auth\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:27:59.684542 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.684289 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74900825-b343-4085-aee1-d3e8c0bf2064-cert\") pod \"ingress-canary-mwh52\" (UID: \"74900825-b343-4085-aee1-d3e8c0bf2064\") " pod="openshift-ingress-canary/ingress-canary-mwh52" Jul 07 17:27:59.684542 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.684342 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b398fe77-3362-4eaa-b811-95a3d049fcb5-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-mr9z9\" (UID: \"b398fe77-3362-4eaa-b811-95a3d049fcb5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9" Jul 07 17:27:59.684542 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.684356 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8eea6af9-b0e3-4bf5-9ee8-aa45037566a3-serving-cert\") pod \"console-operator-68b757fb67-nxsw4\" (UID: \"8eea6af9-b0e3-4bf5-9ee8-aa45037566a3\") " pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:27:59.684542 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.684433 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-default-certificate\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:27:59.684813 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.684693 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4a2ea0f-ff58-4688-b947-46d4352afc62-image-registry-private-configuration\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.684862 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.684810 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c72df9fa-884f-4fac-8557-199256e8c242-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-hrfnv\" (UID: \"c72df9fa-884f-4fac-8557-199256e8c242\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv" Jul 07 17:27:59.685259 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.685240 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4a2ea0f-ff58-4688-b947-46d4352afc62-installation-pull-secrets\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.699294 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.699265 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-bound-sa-token\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.704854 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.704255 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7kvh\" (UniqueName: \"kubernetes.io/projected/b6c8843e-708e-4e0d-a538-e35c93a4beec-kube-api-access-p7kvh\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:27:59.704854 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.704787 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nfjz\" (UniqueName: \"kubernetes.io/projected/2308b91e-a103-451e-b06f-fc5bb0b50c97-kube-api-access-6nfjz\") pod \"cluster-monitoring-operator-64786d9cbb-52xr5\" (UID: \"2308b91e-a103-451e-b06f-fc5bb0b50c97\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:27:59.706041 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.706020 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77n5f\" (UniqueName: \"kubernetes.io/projected/c72df9fa-884f-4fac-8557-199256e8c242-kube-api-access-77n5f\") pod \"service-ca-operator-5d46d9ffd4-hrfnv\" (UID: \"c72df9fa-884f-4fac-8557-199256e8c242\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv" Jul 07 17:27:59.706755 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.706656 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2vwt\" (UniqueName: \"kubernetes.io/projected/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-kube-api-access-z2vwt\") pod \"cluster-samples-operator-56fcf86bf-4l42k\" (UID: \"8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" Jul 07 17:27:59.706903 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.706875 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gz85\" (UniqueName: \"kubernetes.io/projected/b398fe77-3362-4eaa-b811-95a3d049fcb5-kube-api-access-2gz85\") pod \"kube-storage-version-migrator-operator-846bf668f7-mr9z9\" (UID: \"b398fe77-3362-4eaa-b811-95a3d049fcb5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9" Jul 07 17:27:59.707749 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.707721 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t2mf\" (UniqueName: \"kubernetes.io/projected/74900825-b343-4085-aee1-d3e8c0bf2064-kube-api-access-5t2mf\") pod \"ingress-canary-mwh52\" (UID: \"74900825-b343-4085-aee1-d3e8c0bf2064\") " pod="openshift-ingress-canary/ingress-canary-mwh52" Jul 07 17:27:59.708297 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.708272 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cs4f\" (UniqueName: \"kubernetes.io/projected/9dcbb03e-a464-408b-a012-8fc6a386ef16-kube-api-access-6cs4f\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:27:59.708389 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.708325 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgkn8\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-kube-api-access-qgkn8\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:27:59.708946 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.708917 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qrzv\" (UniqueName: \"kubernetes.io/projected/b6d93f73-0d7b-44c8-bd96-b707875dd0af-kube-api-access-5qrzv\") pod \"volume-data-source-validator-5d456fdbcd-ds5pp\" (UID: \"b6d93f73-0d7b-44c8-bd96-b707875dd0af\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds5pp" Jul 07 17:27:59.709084 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.708997 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h75qf\" (UniqueName: \"kubernetes.io/projected/8eea6af9-b0e3-4bf5-9ee8-aa45037566a3-kube-api-access-h75qf\") pod \"console-operator-68b757fb67-nxsw4\" (UID: \"8eea6af9-b0e3-4bf5-9ee8-aa45037566a3\") " pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:27:59.709481 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.709455 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kqnv\" (UniqueName: \"kubernetes.io/projected/15b2d389-16ed-4641-bffb-91298e7516c2-kube-api-access-6kqnv\") pod \"insights-operator-5c847dcfc4-5gff8\" (UID: \"15b2d389-16ed-4641-bffb-91298e7516c2\") " pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.710105 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.710066 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:27:59.776825 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.776785 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds5pp" Jul 07 17:27:59.787191 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.787159 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv" Jul 07 17:27:59.793512 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.793478 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9" Jul 07 17:27:59.809132 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.809092 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:27:59.817519 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.817482 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" Jul 07 17:27:59.828588 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.828559 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mwh52" Jul 07 17:27:59.858368 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:27:59.858278 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:00.054703 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.054282 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9"] Jul 07 17:28:00.058410 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:00.057941 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb398fe77_3362_4eaa_b811_95a3d049fcb5.slice/crio-658e121a67ab15a64c438f417a929c30aa7fe46ea324637a1ae8b8a40d0526b4 WatchSource:0}: Error finding container 658e121a67ab15a64c438f417a929c30aa7fe46ea324637a1ae8b8a40d0526b4: Status 404 returned error can't find the container with id 658e121a67ab15a64c438f417a929c30aa7fe46ea324637a1ae8b8a40d0526b4 Jul 07 17:28:00.060171 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.060021 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-nxsw4"] Jul 07 17:28:00.062858 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:00.062819 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eea6af9_b0e3_4bf5_9ee8_aa45037566a3.slice/crio-546d3a99c17f10a68fbfc0154d185aee7ff02de094a4713a688e3858ec73f3d1 WatchSource:0}: Error finding container 546d3a99c17f10a68fbfc0154d185aee7ff02de094a4713a688e3858ec73f3d1: Status 404 returned error can't find the container with id 546d3a99c17f10a68fbfc0154d185aee7ff02de094a4713a688e3858ec73f3d1 Jul 07 17:28:00.076090 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.076060 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds5pp"] Jul 07 17:28:00.081073 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:00.081046 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d93f73_0d7b_44c8_bd96_b707875dd0af.slice/crio-97784a741822001502c7e27b808555b522caa04eeccb9f0e4f95981957936ac6 WatchSource:0}: Error finding container 97784a741822001502c7e27b808555b522caa04eeccb9f0e4f95981957936ac6: Status 404 returned error can't find the container with id 97784a741822001502c7e27b808555b522caa04eeccb9f0e4f95981957936ac6 Jul 07 17:28:00.092043 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.092020 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv"] Jul 07 17:28:00.093850 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.093823 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-5gff8"] Jul 07 17:28:00.096109 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:00.096052 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc72df9fa_884f_4fac_8557_199256e8c242.slice/crio-ac1583f98fb44d87d9e9e25c351d99438418a106e1a80ff4c83aa16759254e03 WatchSource:0}: Error finding container ac1583f98fb44d87d9e9e25c351d99438418a106e1a80ff4c83aa16759254e03: Status 404 returned error can't find the container with id ac1583f98fb44d87d9e9e25c351d99438418a106e1a80ff4c83aa16759254e03 Jul 07 17:28:00.096957 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.096652 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mwh52"] Jul 07 17:28:00.099420 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:00.099386 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15b2d389_16ed_4641_bffb_91298e7516c2.slice/crio-8ad0614a0cbd2807a52157f6f32756fc37ec20fec2d5757f1c5fe7636b3b35b2 WatchSource:0}: Error finding container 8ad0614a0cbd2807a52157f6f32756fc37ec20fec2d5757f1c5fe7636b3b35b2: Status 404 returned error can't find the container with id 8ad0614a0cbd2807a52157f6f32756fc37ec20fec2d5757f1c5fe7636b3b35b2 Jul 07 17:28:00.188701 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.188566 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:28:00.188701 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.188618 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:28:00.188701 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.188646 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:00.188701 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.188672 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52xr5\" (UID: \"2308b91e-a103-451e-b06f-fc5bb0b50c97\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:28:00.189061 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.188715 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:00.189061 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.188748 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l42k\" (UID: \"8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" Jul 07 17:28:00.189061 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:00.188902 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 17:28:00.189061 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:00.188989 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls podName:8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b nodeName:}" failed. No retries permitted until 2026-07-07 17:28:01.188948878 +0000 UTC m=+37.070585386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-4l42k" (UID: "8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b") : secret "samples-operator-tls" not found Jul 07 17:28:00.189407 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:00.189386 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:28:00.189407 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:00.189407 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fc965b778-q5jc7: secret "image-registry-tls" not found Jul 07 17:28:00.189538 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:00.189453 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls podName:e4a2ea0f-ff58-4688-b947-46d4352afc62 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:01.189437049 +0000 UTC m=+37.071073569 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls") pod "image-registry-7fc965b778-q5jc7" (UID: "e4a2ea0f-ff58-4688-b947-46d4352afc62") : secret "image-registry-tls" not found Jul 07 17:28:00.189538 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:00.189512 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:28:00.189645 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:00.189553 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls podName:9dcbb03e-a464-408b-a012-8fc6a386ef16 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:01.189533453 +0000 UTC m=+37.071169962 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls") pod "dns-default-n4h5f" (UID: "9dcbb03e-a464-408b-a012-8fc6a386ef16") : secret "dns-default-metrics-tls" not found Jul 07 17:28:00.189645 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:00.189606 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 17:28:00.189645 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:00.189638 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs podName:b6c8843e-708e-4e0d-a538-e35c93a4beec nodeName:}" failed. No retries permitted until 2026-07-07 17:28:01.189627253 +0000 UTC m=+37.071263772 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs") pod "router-default-7fbfb58cb7-68cb8" (UID: "b6c8843e-708e-4e0d-a538-e35c93a4beec") : secret "router-metrics-certs-default" not found Jul 07 17:28:00.189789 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:00.189688 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 17:28:00.189789 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:00.189721 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls podName:2308b91e-a103-451e-b06f-fc5bb0b50c97 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:01.189710605 +0000 UTC m=+37.071347112 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-52xr5" (UID: "2308b91e-a103-451e-b06f-fc5bb0b50c97") : secret "cluster-monitoring-operator-tls" not found Jul 07 17:28:00.189789 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:00.189787 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle podName:b6c8843e-708e-4e0d-a538-e35c93a4beec nodeName:}" failed. No retries permitted until 2026-07-07 17:28:01.189777759 +0000 UTC m=+37.071414267 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle") pod "router-default-7fbfb58cb7-68cb8" (UID: "b6c8843e-708e-4e0d-a538-e35c93a4beec") : configmap references non-existent config key: service-ca.crt Jul 07 17:28:00.375832 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.375795 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:00.479600 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.479519 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:00.878882 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.878841 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" event={"ID":"15b2d389-16ed-4641-bffb-91298e7516c2","Type":"ContainerStarted","Data":"8ad0614a0cbd2807a52157f6f32756fc37ec20fec2d5757f1c5fe7636b3b35b2"} Jul 07 17:28:00.880424 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.880337 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" event={"ID":"8eea6af9-b0e3-4bf5-9ee8-aa45037566a3","Type":"ContainerStarted","Data":"546d3a99c17f10a68fbfc0154d185aee7ff02de094a4713a688e3858ec73f3d1"} Jul 07 17:28:00.882324 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.882261 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv" event={"ID":"c72df9fa-884f-4fac-8557-199256e8c242","Type":"ContainerStarted","Data":"ac1583f98fb44d87d9e9e25c351d99438418a106e1a80ff4c83aa16759254e03"} Jul 07 17:28:00.884291 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.884195 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mwh52" event={"ID":"74900825-b343-4085-aee1-d3e8c0bf2064","Type":"ContainerStarted","Data":"5cddafb4abd662d99d3dc6d5d14dca635c9785c2b8662bf2e77c4994ab2ef4e5"} Jul 07 17:28:00.886127 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.886051 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9" event={"ID":"b398fe77-3362-4eaa-b811-95a3d049fcb5","Type":"ContainerStarted","Data":"658e121a67ab15a64c438f417a929c30aa7fe46ea324637a1ae8b8a40d0526b4"} Jul 07 17:28:00.887498 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:00.887436 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds5pp" event={"ID":"b6d93f73-0d7b-44c8-bd96-b707875dd0af","Type":"ContainerStarted","Data":"97784a741822001502c7e27b808555b522caa04eeccb9f0e4f95981957936ac6"} Jul 07 17:28:01.201829 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:01.200435 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:28:01.201829 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:01.200553 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:28:01.201829 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:01.200588 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:01.201829 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:01.200620 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52xr5\" (UID: \"2308b91e-a103-451e-b06f-fc5bb0b50c97\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:28:01.201829 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:01.200671 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:01.201829 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:01.200704 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l42k\" (UID: \"8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" Jul 07 17:28:01.201829 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:01.200879 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 17:28:01.201829 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:01.200940 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls podName:8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b nodeName:}" failed. No retries permitted until 2026-07-07 17:28:03.20092138 +0000 UTC m=+39.082557892 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-4l42k" (UID: "8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b") : secret "samples-operator-tls" not found Jul 07 17:28:01.201829 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:01.201370 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:28:01.201829 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:01.201387 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fc965b778-q5jc7: secret "image-registry-tls" not found Jul 07 17:28:01.201829 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:01.201435 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls podName:e4a2ea0f-ff58-4688-b947-46d4352afc62 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:03.201419426 +0000 UTC m=+39.083055936 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls") pod "image-registry-7fc965b778-q5jc7" (UID: "e4a2ea0f-ff58-4688-b947-46d4352afc62") : secret "image-registry-tls" not found Jul 07 17:28:01.201829 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:01.201494 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:28:01.201829 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:01.201523 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls podName:9dcbb03e-a464-408b-a012-8fc6a386ef16 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:03.201512912 +0000 UTC m=+39.083149418 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls") pod "dns-default-n4h5f" (UID: "9dcbb03e-a464-408b-a012-8fc6a386ef16") : secret "dns-default-metrics-tls" not found Jul 07 17:28:01.201829 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:01.201574 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 17:28:01.201829 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:01.201603 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs podName:b6c8843e-708e-4e0d-a538-e35c93a4beec nodeName:}" failed. No retries permitted until 2026-07-07 17:28:03.201593393 +0000 UTC m=+39.083229904 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs") pod "router-default-7fbfb58cb7-68cb8" (UID: "b6c8843e-708e-4e0d-a538-e35c93a4beec") : secret "router-metrics-certs-default" not found Jul 07 17:28:01.201829 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:01.201653 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 17:28:01.202766 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:01.201682 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls podName:2308b91e-a103-451e-b06f-fc5bb0b50c97 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:03.201672922 +0000 UTC m=+39.083309431 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-52xr5" (UID: "2308b91e-a103-451e-b06f-fc5bb0b50c97") : secret "cluster-monitoring-operator-tls" not found Jul 07 17:28:01.202766 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:01.201747 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle podName:b6c8843e-708e-4e0d-a538-e35c93a4beec nodeName:}" failed. No retries permitted until 2026-07-07 17:28:03.201736942 +0000 UTC m=+39.083373451 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle") pod "router-default-7fbfb58cb7-68cb8" (UID: "b6c8843e-708e-4e0d-a538-e35c93a4beec") : configmap references non-existent config key: service-ca.crt Jul 07 17:28:03.223422 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:03.223378 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:28:03.223422 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:03.223437 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:28:03.224001 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:03.223475 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:03.224001 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:03.223509 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52xr5\" (UID: \"2308b91e-a103-451e-b06f-fc5bb0b50c97\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:28:03.224001 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:03.223544 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:03.224001 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:03.223563 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l42k\" (UID: \"8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" Jul 07 17:28:03.224001 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:03.223577 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:28:03.224001 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:03.223605 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fc965b778-q5jc7: secret "image-registry-tls" not found Jul 07 17:28:03.224001 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:03.223677 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls podName:e4a2ea0f-ff58-4688-b947-46d4352afc62 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:07.223654435 +0000 UTC m=+43.105290942 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls") pod "image-registry-7fc965b778-q5jc7" (UID: "e4a2ea0f-ff58-4688-b947-46d4352afc62") : secret "image-registry-tls" not found Jul 07 17:28:03.224001 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:03.223680 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 17:28:03.224001 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:03.223711 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:28:03.224001 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:03.223720 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls podName:8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b nodeName:}" failed. No retries permitted until 2026-07-07 17:28:07.223709479 +0000 UTC m=+43.105345987 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-4l42k" (UID: "8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b") : secret "samples-operator-tls" not found Jul 07 17:28:03.224001 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:03.223761 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls podName:9dcbb03e-a464-408b-a012-8fc6a386ef16 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:07.223749707 +0000 UTC m=+43.105386212 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls") pod "dns-default-n4h5f" (UID: "9dcbb03e-a464-408b-a012-8fc6a386ef16") : secret "dns-default-metrics-tls" not found Jul 07 17:28:03.224001 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:03.223795 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 17:28:03.224001 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:03.223814 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs podName:b6c8843e-708e-4e0d-a538-e35c93a4beec nodeName:}" failed. No retries permitted until 2026-07-07 17:28:07.223808325 +0000 UTC m=+43.105444829 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs") pod "router-default-7fbfb58cb7-68cb8" (UID: "b6c8843e-708e-4e0d-a538-e35c93a4beec") : secret "router-metrics-certs-default" not found Jul 07 17:28:03.224001 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:03.223844 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 17:28:03.224001 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:03.223863 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls podName:2308b91e-a103-451e-b06f-fc5bb0b50c97 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:07.223857029 +0000 UTC m=+43.105493533 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-52xr5" (UID: "2308b91e-a103-451e-b06f-fc5bb0b50c97") : secret "cluster-monitoring-operator-tls" not found Jul 07 17:28:03.224692 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:03.223919 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle podName:b6c8843e-708e-4e0d-a538-e35c93a4beec nodeName:}" failed. No retries permitted until 2026-07-07 17:28:07.223912951 +0000 UTC m=+43.105549456 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle") pod "router-default-7fbfb58cb7-68cb8" (UID: "b6c8843e-708e-4e0d-a538-e35c93a4beec") : configmap references non-existent config key: service-ca.crt Jul 07 17:28:04.220199 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:04.220155 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:04.376439 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:04.376408 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:04.486214 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:04.486141 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:04.763835 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:04.763728 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:04.866351 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:04.866309 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:05.202811 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:05.202775 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:05.280832 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:05.280770 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:06.146251 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:06.146205 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:06.213331 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:06.213287 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:06.343853 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:06.343818 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-qjzjz"] Jul 07 17:28:06.368957 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:06.368925 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-qjzjz"] Jul 07 17:28:06.369108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:06.369058 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-qjzjz" Jul 07 17:28:06.372242 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:06.372223 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-jkskp\"" Jul 07 17:28:06.554209 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:06.554120 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mxrp\" (UniqueName: \"kubernetes.io/projected/af3cc707-9ed1-4242-a6e1-0d7606a935e7-kube-api-access-5mxrp\") pod \"network-check-source-989f8b8dc-qjzjz\" (UID: \"af3cc707-9ed1-4242-a6e1-0d7606a935e7\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-qjzjz" Jul 07 17:28:06.654622 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:06.654585 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mxrp\" (UniqueName: \"kubernetes.io/projected/af3cc707-9ed1-4242-a6e1-0d7606a935e7-kube-api-access-5mxrp\") pod \"network-check-source-989f8b8dc-qjzjz\" (UID: \"af3cc707-9ed1-4242-a6e1-0d7606a935e7\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-qjzjz" Jul 07 17:28:06.668478 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:06.668440 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mxrp\" (UniqueName: \"kubernetes.io/projected/af3cc707-9ed1-4242-a6e1-0d7606a935e7-kube-api-access-5mxrp\") pod \"network-check-source-989f8b8dc-qjzjz\" (UID: \"af3cc707-9ed1-4242-a6e1-0d7606a935e7\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-qjzjz" Jul 07 17:28:06.678144 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:06.678113 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-qjzjz" Jul 07 17:28:07.060596 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:07.060565 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:07.160862 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:07.160822 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:07.260418 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:07.260383 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:28:07.260418 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:07.260428 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:07.260639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:07.260458 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52xr5\" (UID: \"2308b91e-a103-451e-b06f-fc5bb0b50c97\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:28:07.260639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:07.260506 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:07.260639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:07.260537 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l42k\" (UID: \"8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" Jul 07 17:28:07.260639 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:07.260548 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:28:07.260639 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:07.260610 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls podName:9dcbb03e-a464-408b-a012-8fc6a386ef16 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:15.260592331 +0000 UTC m=+51.142228839 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls") pod "dns-default-n4h5f" (UID: "9dcbb03e-a464-408b-a012-8fc6a386ef16") : secret "dns-default-metrics-tls" not found Jul 07 17:28:07.260639 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:07.260623 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 17:28:07.260862 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:07.260655 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 17:28:07.260862 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:07.260667 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:28:07.260862 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:07.260712 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs podName:b6c8843e-708e-4e0d-a538-e35c93a4beec nodeName:}" failed. No retries permitted until 2026-07-07 17:28:15.2606937 +0000 UTC m=+51.142330222 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs") pod "router-default-7fbfb58cb7-68cb8" (UID: "b6c8843e-708e-4e0d-a538-e35c93a4beec") : secret "router-metrics-certs-default" not found Jul 07 17:28:07.260862 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:07.260729 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls podName:8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b nodeName:}" failed. No retries permitted until 2026-07-07 17:28:15.260721547 +0000 UTC m=+51.142358055 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-4l42k" (UID: "8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b") : secret "samples-operator-tls" not found Jul 07 17:28:07.260862 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:07.260744 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 17:28:07.260862 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:07.260774 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls podName:2308b91e-a103-451e-b06f-fc5bb0b50c97 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:15.260763576 +0000 UTC m=+51.142400085 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-52xr5" (UID: "2308b91e-a103-451e-b06f-fc5bb0b50c97") : secret "cluster-monitoring-operator-tls" not found Jul 07 17:28:07.260862 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:07.260794 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle podName:b6c8843e-708e-4e0d-a538-e35c93a4beec nodeName:}" failed. No retries permitted until 2026-07-07 17:28:15.260786207 +0000 UTC m=+51.142422712 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle") pod "router-default-7fbfb58cb7-68cb8" (UID: "b6c8843e-708e-4e0d-a538-e35c93a4beec") : configmap references non-existent config key: service-ca.crt Jul 07 17:28:07.260862 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:07.260824 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:28:07.260862 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:07.260834 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fc965b778-q5jc7: secret "image-registry-tls" not found Jul 07 17:28:07.260862 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:07.260866 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls podName:e4a2ea0f-ff58-4688-b947-46d4352afc62 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:15.260855138 +0000 UTC m=+51.142491643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls") pod "image-registry-7fc965b778-q5jc7" (UID: "e4a2ea0f-ff58-4688-b947-46d4352afc62") : secret "image-registry-tls" not found Jul 07 17:28:08.231395 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.231358 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:08.374102 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.374076 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-qjzjz"] Jul 07 17:28:08.380294 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:08.380272 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf3cc707_9ed1_4242_a6e1_0d7606a935e7.slice/crio-072745f539b405f7fdcbbd012a2c897247f28e90efa09d9211511c454c1ac109 WatchSource:0}: Error finding container 072745f539b405f7fdcbbd012a2c897247f28e90efa09d9211511c454c1ac109: Status 404 returned error can't find the container with id 072745f539b405f7fdcbbd012a2c897247f28e90efa09d9211511c454c1ac109 Jul 07 17:28:08.912192 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.912099 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds5pp" event={"ID":"b6d93f73-0d7b-44c8-bd96-b707875dd0af","Type":"ContainerStarted","Data":"5845976bccda4e36c2009ed0dfffc32df79a84b6ef48ad0679f3d48d1dce9dd8"} Jul 07 17:28:08.913399 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.913366 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-qjzjz" event={"ID":"af3cc707-9ed1-4242-a6e1-0d7606a935e7","Type":"ContainerStarted","Data":"072745f539b405f7fdcbbd012a2c897247f28e90efa09d9211511c454c1ac109"} Jul 07 17:28:08.914897 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.914871 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" event={"ID":"15b2d389-16ed-4641-bffb-91298e7516c2","Type":"ContainerStarted","Data":"0cafaeb6107307fd71df4d4e856079f7935b50479261e78e0248444c563a588c"} Jul 07 17:28:08.916728 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.916707 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/0.log" Jul 07 17:28:08.916834 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.916759 2562 generic.go:356] "Generic (PLEG): container finished" podID="8eea6af9-b0e3-4bf5-9ee8-aa45037566a3" containerID="8cf46df64f81406855eacd9fc15ac8f0f05968b4107b6bca0923fd3fe0553557" exitCode=255 Jul 07 17:28:08.916894 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.916822 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" event={"ID":"8eea6af9-b0e3-4bf5-9ee8-aa45037566a3","Type":"ContainerDied","Data":"8cf46df64f81406855eacd9fc15ac8f0f05968b4107b6bca0923fd3fe0553557"} Jul 07 17:28:08.917641 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.917076 2562 scope.go:117] "RemoveContainer" containerID="8cf46df64f81406855eacd9fc15ac8f0f05968b4107b6bca0923fd3fe0553557" Jul 07 17:28:08.919086 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.919036 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv" event={"ID":"c72df9fa-884f-4fac-8557-199256e8c242","Type":"ContainerStarted","Data":"124a957e9b0a1c82a488be9a025fb5a5d8ffc1310b9efe3479097da8c289c9e7"} Jul 07 17:28:08.922026 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.922003 2562 generic.go:356] "Generic (PLEG): container finished" podID="f81d1bd5-43a4-476a-b1c8-2a637c464bc9" containerID="c43fac571a23f2637a86d75eeb22847329a4bf45046ec97ae2424bbbb27b4a5f" exitCode=0 Jul 07 17:28:08.922133 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.922085 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pc8dw" event={"ID":"f81d1bd5-43a4-476a-b1c8-2a637c464bc9","Type":"ContainerDied","Data":"c43fac571a23f2637a86d75eeb22847329a4bf45046ec97ae2424bbbb27b4a5f"} Jul 07 17:28:08.924268 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.923806 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mwh52" event={"ID":"74900825-b343-4085-aee1-d3e8c0bf2064","Type":"ContainerStarted","Data":"9ba7bd528e97e205a828d12871ef113fde8ad8e02b87ecadbbacbc01760f0b2e"} Jul 07 17:28:08.926648 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.926615 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9" event={"ID":"b398fe77-3362-4eaa-b811-95a3d049fcb5","Type":"ContainerStarted","Data":"65bb35b82a264e6879d5883364aef117ba9a46ec2e9e780f7ae32d25a0b8cba7"} Jul 07 17:28:08.931839 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.931795 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds5pp" podStartSLOduration=4.7807609410000005 podStartE2EDuration="12.931778024s" totalImagesPullingTime="8.151017083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:27:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:08.931251152 +0000 UTC m=+44.812887678" watchObservedRunningTime="2026-07-07 17:28:08.931778024 +0000 UTC m=+44.813414553" Jul 07 17:28:08.950664 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.950608 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mwh52" podStartSLOduration=1.812767138 podStartE2EDuration="9.950586991s" totalImagesPullingTime="8.137819853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:08.949634704 +0000 UTC m=+44.831271232" watchObservedRunningTime="2026-07-07 17:28:08.950586991 +0000 UTC m=+44.832223519" Jul 07 17:28:08.999034 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:08.998954 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" podStartSLOduration=14.866992555 podStartE2EDuration="22.998930117s" totalImagesPullingTime="8.131937562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:08.998366082 +0000 UTC m=+44.880002609" watchObservedRunningTime="2026-07-07 17:28:08.998930117 +0000 UTC m=+44.880566645" Jul 07 17:28:09.031902 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:09.031822 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:09.053501 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:09.053434 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv" podStartSLOduration=4.918342099 podStartE2EDuration="13.053412374s" totalImagesPullingTime="8.135070275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:27:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:09.051324677 +0000 UTC m=+44.932961205" watchObservedRunningTime="2026-07-07 17:28:09.053412374 +0000 UTC m=+44.935048897" Jul 07 17:28:09.071787 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:09.071741 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9" podStartSLOduration=4.897691372 podStartE2EDuration="13.071710543s" totalImagesPullingTime="8.174019171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:27:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:09.070276461 +0000 UTC m=+44.951912981" watchObservedRunningTime="2026-07-07 17:28:09.071710543 +0000 UTC m=+44.953347118" Jul 07 17:28:09.104516 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:09.104438 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:09.809657 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:09.809620 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:28:09.810479 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:09.809683 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:28:09.850484 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:09.850422 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:09.934830 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:09.934790 2562 generic.go:356] "Generic (PLEG): container finished" podID="f81d1bd5-43a4-476a-b1c8-2a637c464bc9" containerID="9365f8b917ca463053a31f9d702e7e8f730c30fe7a3c0d81d116ff6ccaa0a07d" exitCode=0 Jul 07 17:28:09.935035 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:09.934871 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pc8dw" event={"ID":"f81d1bd5-43a4-476a-b1c8-2a637c464bc9","Type":"ContainerDied","Data":"9365f8b917ca463053a31f9d702e7e8f730c30fe7a3c0d81d116ff6ccaa0a07d"} Jul 07 17:28:09.936911 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:09.936888 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 17:28:09.937308 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:09.937290 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/0.log" Jul 07 17:28:09.937369 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:09.937329 2562 generic.go:356] "Generic (PLEG): container finished" podID="8eea6af9-b0e3-4bf5-9ee8-aa45037566a3" containerID="68c85c129c25217e539a750a2ac3facbfbad107a75e9153885e838b2b39dc0a1" exitCode=255 Jul 07 17:28:09.938015 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:09.938001 2562 scope.go:117] "RemoveContainer" containerID="68c85c129c25217e539a750a2ac3facbfbad107a75e9153885e838b2b39dc0a1" Jul 07 17:28:09.938175 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:09.938155 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-nxsw4_openshift-console-operator(8eea6af9-b0e3-4bf5-9ee8-aa45037566a3)\"" pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" podUID="8eea6af9-b0e3-4bf5-9ee8-aa45037566a3" Jul 07 17:28:09.938227 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:09.938198 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" event={"ID":"8eea6af9-b0e3-4bf5-9ee8-aa45037566a3","Type":"ContainerDied","Data":"68c85c129c25217e539a750a2ac3facbfbad107a75e9153885e838b2b39dc0a1"} Jul 07 17:28:09.938258 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:09.938247 2562 scope.go:117] "RemoveContainer" containerID="8cf46df64f81406855eacd9fc15ac8f0f05968b4107b6bca0923fd3fe0553557" Jul 07 17:28:10.350823 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.350739 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-25z9v"] Jul 07 17:28:10.357205 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.357171 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:10.359952 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.359922 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 17:28:10.359952 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.359945 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8ztdc\"" Jul 07 17:28:10.360176 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.359933 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 17:28:10.368568 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.368537 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-25z9v"] Jul 07 17:28:10.392284 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.392248 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn5r6\" (UniqueName: \"kubernetes.io/projected/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-api-access-sn5r6\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:10.392463 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.392308 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/19788e63-8c11-4c84-9998-c2e28b6c5908-data-volume\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:10.392463 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.392414 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:10.392463 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.392453 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/19788e63-8c11-4c84-9998-c2e28b6c5908-crio-socket\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:10.392605 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.392477 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:10.444095 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.444052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:10.493408 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.493371 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:10.493610 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.493446 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/19788e63-8c11-4c84-9998-c2e28b6c5908-crio-socket\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:10.493610 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.493476 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:10.493610 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:10.493542 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 17:28:10.493610 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:10.493577 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 17:28:10.493802 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:10.493625 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls podName:19788e63-8c11-4c84-9998-c2e28b6c5908 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:10.993606014 +0000 UTC m=+46.875242531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls") pod "insights-runtime-extractor-25z9v" (UID: "19788e63-8c11-4c84-9998-c2e28b6c5908") : secret "insights-runtime-extractor-tls" not found Jul 07 17:28:10.493802 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:10.493644 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-rbac-proxy-cm podName:19788e63-8c11-4c84-9998-c2e28b6c5908 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:10.99363508 +0000 UTC m=+46.875271585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-rbac-proxy-cm") pod "insights-runtime-extractor-25z9v" (UID: "19788e63-8c11-4c84-9998-c2e28b6c5908") : configmap "kube-rbac-proxy" not found Jul 07 17:28:10.493802 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.493708 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/19788e63-8c11-4c84-9998-c2e28b6c5908-crio-socket\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:10.493802 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.493757 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn5r6\" (UniqueName: \"kubernetes.io/projected/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-api-access-sn5r6\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:10.494032 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.493802 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/19788e63-8c11-4c84-9998-c2e28b6c5908-data-volume\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:10.494216 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.494195 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/19788e63-8c11-4c84-9998-c2e28b6c5908-data-volume\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:10.500357 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.500329 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-cg5hg"] Jul 07 17:28:10.509589 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.509522 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn5r6\" (UniqueName: \"kubernetes.io/projected/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-api-access-sn5r6\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:10.517242 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.517193 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-cg5hg"] Jul 07 17:28:10.517401 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.517321 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cg5hg" Jul 07 17:28:10.521387 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.521352 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 07 17:28:10.521576 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.521558 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ljqr7\"" Jul 07 17:28:10.521776 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.521763 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 07 17:28:10.545457 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.545371 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:10.595092 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.595056 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txpsf\" (UniqueName: \"kubernetes.io/projected/25624a89-9856-4260-bccc-02578c46bc23-kube-api-access-txpsf\") pod \"migrator-d57d58cc9-cg5hg\" (UID: \"25624a89-9856-4260-bccc-02578c46bc23\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cg5hg" Jul 07 17:28:10.644026 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.643923 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:10.696457 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.696420 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txpsf\" (UniqueName: \"kubernetes.io/projected/25624a89-9856-4260-bccc-02578c46bc23-kube-api-access-txpsf\") pod \"migrator-d57d58cc9-cg5hg\" (UID: \"25624a89-9856-4260-bccc-02578c46bc23\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cg5hg" Jul 07 17:28:10.707199 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.707162 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txpsf\" (UniqueName: \"kubernetes.io/projected/25624a89-9856-4260-bccc-02578c46bc23-kube-api-access-txpsf\") pod \"migrator-d57d58cc9-cg5hg\" (UID: \"25624a89-9856-4260-bccc-02578c46bc23\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cg5hg" Jul 07 17:28:10.736913 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.736874 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:10.832064 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.832025 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cg5hg" Jul 07 17:28:10.945131 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.945107 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 17:28:10.945565 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.945545 2562 scope.go:117] "RemoveContainer" containerID="68c85c129c25217e539a750a2ac3facbfbad107a75e9153885e838b2b39dc0a1" Jul 07 17:28:10.945766 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:10.945742 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-nxsw4_openshift-console-operator(8eea6af9-b0e3-4bf5-9ee8-aa45037566a3)\"" pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" podUID="8eea6af9-b0e3-4bf5-9ee8-aa45037566a3" Jul 07 17:28:10.958112 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.950131 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pc8dw" event={"ID":"f81d1bd5-43a4-476a-b1c8-2a637c464bc9","Type":"ContainerStarted","Data":"2e30ab7e019bfa6cd1a658ad606cc825c85964a9c3d03231bec1ec14b3752764"} Jul 07 17:28:10.990842 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:10.990787 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pc8dw" podStartSLOduration=13.503425503999999 podStartE2EDuration="46.990769652s" totalImagesPullingTime="33.487344148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:27:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:10.988780118 +0000 UTC m=+46.870416646" watchObservedRunningTime="2026-07-07 17:28:10.990769652 +0000 UTC m=+46.872406179" Jul 07 17:28:11.000197 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:11.000160 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:11.000375 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:11.000239 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:11.000375 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:11.000323 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 17:28:11.000375 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:11.000346 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 17:28:11.000375 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:11.000371 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-rbac-proxy-cm podName:19788e63-8c11-4c84-9998-c2e28b6c5908 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:12.00035858 +0000 UTC m=+47.881995085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-rbac-proxy-cm") pod "insights-runtime-extractor-25z9v" (UID: "19788e63-8c11-4c84-9998-c2e28b6c5908") : configmap "kube-rbac-proxy" not found Jul 07 17:28:11.000577 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:11.000405 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls podName:19788e63-8c11-4c84-9998-c2e28b6c5908 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:12.000388383 +0000 UTC m=+47.882024907 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls") pod "insights-runtime-extractor-25z9v" (UID: "19788e63-8c11-4c84-9998-c2e28b6c5908") : secret "insights-runtime-extractor-tls" not found Jul 07 17:28:11.007905 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:11.007878 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-cg5hg"] Jul 07 17:28:11.010349 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:11.010317 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25624a89_9856_4260_bccc_02578c46bc23.slice/crio-3ff08d141129e84bf34330f161c56b5c45351ce7c2f7f29292b9b54d675b104d WatchSource:0}: Error finding container 3ff08d141129e84bf34330f161c56b5c45351ce7c2f7f29292b9b54d675b104d: Status 404 returned error can't find the container with id 3ff08d141129e84bf34330f161c56b5c45351ce7c2f7f29292b9b54d675b104d Jul 07 17:28:11.145646 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:11.145602 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:11.221689 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:11.221585 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:11.322590 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:11.322550 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:11.422920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:11.422879 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:11.750605 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:11.750570 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hsg2d_dfec4ce5-3bf7-4600-b522-9daca21cf8c8/dns-node-resolver/0.log" Jul 07 17:28:11.956145 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:11.956102 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cg5hg" event={"ID":"25624a89-9856-4260-bccc-02578c46bc23","Type":"ContainerStarted","Data":"3ff08d141129e84bf34330f161c56b5c45351ce7c2f7f29292b9b54d675b104d"} Jul 07 17:28:11.956574 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:11.956460 2562 scope.go:117] "RemoveContainer" containerID="68c85c129c25217e539a750a2ac3facbfbad107a75e9153885e838b2b39dc0a1" Jul 07 17:28:11.956634 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:11.956615 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-nxsw4_openshift-console-operator(8eea6af9-b0e3-4bf5-9ee8-aa45037566a3)\"" pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" podUID="8eea6af9-b0e3-4bf5-9ee8-aa45037566a3" Jul 07 17:28:12.011572 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:12.011477 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:12.011572 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:12.011545 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:12.012011 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:12.011988 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 17:28:12.012205 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:12.012059 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls podName:19788e63-8c11-4c84-9998-c2e28b6c5908 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:14.012038741 +0000 UTC m=+49.893675253 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls") pod "insights-runtime-extractor-25z9v" (UID: "19788e63-8c11-4c84-9998-c2e28b6c5908") : secret "insights-runtime-extractor-tls" not found Jul 07 17:28:12.012734 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:12.012716 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 17:28:12.012810 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:12.012760 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-rbac-proxy-cm podName:19788e63-8c11-4c84-9998-c2e28b6c5908 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:14.012748988 +0000 UTC m=+49.894385493 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-rbac-proxy-cm") pod "insights-runtime-extractor-25z9v" (UID: "19788e63-8c11-4c84-9998-c2e28b6c5908") : configmap "kube-rbac-proxy" not found Jul 07 17:28:12.346037 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:12.346005 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fx768_84115d28-35c1-4c55-9d01-c0f85ccfda09/node-ca/0.log" Jul 07 17:28:12.981318 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:12.981277 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xdmr8"] Jul 07 17:28:13.002405 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.002372 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xdmr8"] Jul 07 17:28:13.002677 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.002489 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-xdmr8" Jul 07 17:28:13.005575 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.005307 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 07 17:28:13.005575 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.005307 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 07 17:28:13.005575 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.005351 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 07 17:28:13.006564 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.006542 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-ztn77\"" Jul 07 17:28:13.006709 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.006633 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 07 17:28:13.020807 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.020764 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/4040a9df-bb00-4ad1-8d71-edd0701f8673-signing-cabundle\") pod \"service-ca-758f599c5-xdmr8\" (UID: \"4040a9df-bb00-4ad1-8d71-edd0701f8673\") " pod="openshift-service-ca/service-ca-758f599c5-xdmr8" Jul 07 17:28:13.020993 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.020820 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp9m4\" (UniqueName: \"kubernetes.io/projected/4040a9df-bb00-4ad1-8d71-edd0701f8673-kube-api-access-xp9m4\") pod \"service-ca-758f599c5-xdmr8\" (UID: \"4040a9df-bb00-4ad1-8d71-edd0701f8673\") " pod="openshift-service-ca/service-ca-758f599c5-xdmr8" Jul 07 17:28:13.020993 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.020936 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/4040a9df-bb00-4ad1-8d71-edd0701f8673-signing-key\") pod \"service-ca-758f599c5-xdmr8\" (UID: \"4040a9df-bb00-4ad1-8d71-edd0701f8673\") " pod="openshift-service-ca/service-ca-758f599c5-xdmr8" Jul 07 17:28:13.032566 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.032526 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:13.122266 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.122225 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/4040a9df-bb00-4ad1-8d71-edd0701f8673-signing-key\") pod \"service-ca-758f599c5-xdmr8\" (UID: \"4040a9df-bb00-4ad1-8d71-edd0701f8673\") " pod="openshift-service-ca/service-ca-758f599c5-xdmr8" Jul 07 17:28:13.122439 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.122384 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/4040a9df-bb00-4ad1-8d71-edd0701f8673-signing-cabundle\") pod \"service-ca-758f599c5-xdmr8\" (UID: \"4040a9df-bb00-4ad1-8d71-edd0701f8673\") " pod="openshift-service-ca/service-ca-758f599c5-xdmr8" Jul 07 17:28:13.122439 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.122412 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp9m4\" (UniqueName: \"kubernetes.io/projected/4040a9df-bb00-4ad1-8d71-edd0701f8673-kube-api-access-xp9m4\") pod \"service-ca-758f599c5-xdmr8\" (UID: \"4040a9df-bb00-4ad1-8d71-edd0701f8673\") " pod="openshift-service-ca/service-ca-758f599c5-xdmr8" Jul 07 17:28:13.123155 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.123125 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/4040a9df-bb00-4ad1-8d71-edd0701f8673-signing-cabundle\") pod \"service-ca-758f599c5-xdmr8\" (UID: \"4040a9df-bb00-4ad1-8d71-edd0701f8673\") " pod="openshift-service-ca/service-ca-758f599c5-xdmr8" Jul 07 17:28:13.125524 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.125492 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/4040a9df-bb00-4ad1-8d71-edd0701f8673-signing-key\") pod \"service-ca-758f599c5-xdmr8\" (UID: \"4040a9df-bb00-4ad1-8d71-edd0701f8673\") " pod="openshift-service-ca/service-ca-758f599c5-xdmr8" Jul 07 17:28:13.133995 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.133939 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp9m4\" (UniqueName: \"kubernetes.io/projected/4040a9df-bb00-4ad1-8d71-edd0701f8673-kube-api-access-xp9m4\") pod \"service-ca-758f599c5-xdmr8\" (UID: \"4040a9df-bb00-4ad1-8d71-edd0701f8673\") " pod="openshift-service-ca/service-ca-758f599c5-xdmr8" Jul 07 17:28:13.147625 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.147592 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mwh52_74900825-b343-4085-aee1-d3e8c0bf2064/serve-healthcheck-canary/0.log" Jul 07 17:28:13.314989 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.314929 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-xdmr8" Jul 07 17:28:13.469077 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.469034 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xdmr8"] Jul 07 17:28:13.516360 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:13.516287 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4040a9df_bb00_4ad1_8d71_edd0701f8673.slice/crio-db8dcde3fe9e0a9e834e9ea6aaab149a77b6eda01abbdfa4e45051eece32d7a9 WatchSource:0}: Error finding container db8dcde3fe9e0a9e834e9ea6aaab149a77b6eda01abbdfa4e45051eece32d7a9: Status 404 returned error can't find the container with id db8dcde3fe9e0a9e834e9ea6aaab149a77b6eda01abbdfa4e45051eece32d7a9 Jul 07 17:28:13.964829 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.964791 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-qjzjz" event={"ID":"af3cc707-9ed1-4242-a6e1-0d7606a935e7","Type":"ContainerStarted","Data":"fb83f92e7ea520f2c01c8d05e3140836399a660254d147fa7fe48dceaedf5ecd"} Jul 07 17:28:13.966771 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.966735 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cg5hg" event={"ID":"25624a89-9856-4260-bccc-02578c46bc23","Type":"ContainerStarted","Data":"4f21dfacb6f35e19c03ca6cdd288b6d8d82b882d5a52f720166f082e669b9c7f"} Jul 07 17:28:13.966920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.966790 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cg5hg" event={"ID":"25624a89-9856-4260-bccc-02578c46bc23","Type":"ContainerStarted","Data":"5dcaca9e8cd27e24c20189e467a801b13e6c71b1f7739157d2bdd0ec23471e3a"} Jul 07 17:28:13.968297 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.968274 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-xdmr8" event={"ID":"4040a9df-bb00-4ad1-8d71-edd0701f8673","Type":"ContainerStarted","Data":"1742547657f1ff4afa6aa22248414b27b2eb90d5f1f9759605dc5a6d7bc8321e"} Jul 07 17:28:13.968402 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.968316 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-xdmr8" event={"ID":"4040a9df-bb00-4ad1-8d71-edd0701f8673","Type":"ContainerStarted","Data":"db8dcde3fe9e0a9e834e9ea6aaab149a77b6eda01abbdfa4e45051eece32d7a9"} Jul 07 17:28:13.988159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:13.988103 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-qjzjz" podStartSLOduration=3.110443755 podStartE2EDuration="7.988081475s" totalImagesPullingTime="4.87763772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:28:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:13.987713625 +0000 UTC m=+49.869350153" watchObservedRunningTime="2026-07-07 17:28:13.988081475 +0000 UTC m=+49.869718001" Jul 07 17:28:14.012981 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:14.012906 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-cg5hg" podStartSLOduration=1.4927648439999999 podStartE2EDuration="4.012886645s" totalImagesPullingTime="2.520121801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:28:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:14.012412802 +0000 UTC m=+49.894049329" watchObservedRunningTime="2026-07-07 17:28:14.012886645 +0000 UTC m=+49.894523174" Jul 07 17:28:14.030843 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:14.030799 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:14.031183 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:14.031149 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:14.031628 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:14.030920 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 17:28:14.031710 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:14.031691 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls podName:19788e63-8c11-4c84-9998-c2e28b6c5908 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:18.031667563 +0000 UTC m=+53.913304068 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls") pod "insights-runtime-extractor-25z9v" (UID: "19788e63-8c11-4c84-9998-c2e28b6c5908") : secret "insights-runtime-extractor-tls" not found Jul 07 17:28:14.031710 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:14.031282 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 17:28:14.031863 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:14.031760 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-rbac-proxy-cm podName:19788e63-8c11-4c84-9998-c2e28b6c5908 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:18.031741305 +0000 UTC m=+53.913377811 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-rbac-proxy-cm") pod "insights-runtime-extractor-25z9v" (UID: "19788e63-8c11-4c84-9998-c2e28b6c5908") : configmap "kube-rbac-proxy" not found Jul 07 17:28:14.033858 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:14.033830 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:14.038226 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:14.038183 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-xdmr8" podStartSLOduration=2.038168001 podStartE2EDuration="2.038168001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:28:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:14.037841995 +0000 UTC m=+49.919478525" watchObservedRunningTime="2026-07-07 17:28:14.038168001 +0000 UTC m=+49.919804527" Jul 07 17:28:15.341275 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:15.341232 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:28:15.341275 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:15.341284 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:15.341770 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:15.341304 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52xr5\" (UID: \"2308b91e-a103-451e-b06f-fc5bb0b50c97\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:28:15.341770 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:15.341346 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:15.341770 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:15.341368 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l42k\" (UID: \"8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" Jul 07 17:28:15.341770 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:15.341403 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:28:15.341770 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:15.341430 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:28:15.341770 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:15.341469 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls podName:9dcbb03e-a464-408b-a012-8fc6a386ef16 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:31.341453736 +0000 UTC m=+67.223090241 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls") pod "dns-default-n4h5f" (UID: "9dcbb03e-a464-408b-a012-8fc6a386ef16") : secret "dns-default-metrics-tls" not found Jul 07 17:28:15.341770 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:15.341403 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 17:28:15.341770 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:15.341521 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 17:28:15.341770 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:15.341529 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 17:28:15.341770 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:15.341517 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:28:15.341770 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:15.341567 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fc965b778-q5jc7: secret "image-registry-tls" not found Jul 07 17:28:15.341770 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:15.341529 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle podName:b6c8843e-708e-4e0d-a538-e35c93a4beec nodeName:}" failed. No retries permitted until 2026-07-07 17:28:31.341511645 +0000 UTC m=+67.223148163 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle") pod "router-default-7fbfb58cb7-68cb8" (UID: "b6c8843e-708e-4e0d-a538-e35c93a4beec") : configmap references non-existent config key: service-ca.crt Jul 07 17:28:15.341770 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:15.341604 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs podName:b6c8843e-708e-4e0d-a538-e35c93a4beec nodeName:}" failed. No retries permitted until 2026-07-07 17:28:31.341588792 +0000 UTC m=+67.223225297 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs") pod "router-default-7fbfb58cb7-68cb8" (UID: "b6c8843e-708e-4e0d-a538-e35c93a4beec") : secret "router-metrics-certs-default" not found Jul 07 17:28:15.341770 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:15.341624 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls podName:2308b91e-a103-451e-b06f-fc5bb0b50c97 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:31.341617173 +0000 UTC m=+67.223253679 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-52xr5" (UID: "2308b91e-a103-451e-b06f-fc5bb0b50c97") : secret "cluster-monitoring-operator-tls" not found Jul 07 17:28:15.341770 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:15.341634 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls podName:8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b nodeName:}" failed. No retries permitted until 2026-07-07 17:28:31.34162891 +0000 UTC m=+67.223265414 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-4l42k" (UID: "8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b") : secret "samples-operator-tls" not found Jul 07 17:28:15.342249 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:15.341643 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls podName:e4a2ea0f-ff58-4688-b947-46d4352afc62 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:31.341638198 +0000 UTC m=+67.223274703 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls") pod "image-registry-7fc965b778-q5jc7" (UID: "e4a2ea0f-ff58-4688-b947-46d4352afc62") : secret "image-registry-tls" not found Jul 07 17:28:15.504831 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:15.504781 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:15.568592 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:15.568476 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:15.648352 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:15.648252 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:15.802501 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:15.802464 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:15.903362 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:15.903258 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:15.990513 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:15.990483 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:16.067386 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:16.067347 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:16.230946 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:16.230856 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:17.377803 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:17.377765 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:17.494427 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:17.494395 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:17.624538 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:17.624500 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:17.827352 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:17.827317 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:18.044216 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:18.044177 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:18.069592 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:18.069545 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:18.069815 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:18.069613 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:18.069815 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:18.069701 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 17:28:18.069815 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:18.069772 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls podName:19788e63-8c11-4c84-9998-c2e28b6c5908 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:26.069756263 +0000 UTC m=+61.951392768 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls") pod "insights-runtime-extractor-25z9v" (UID: "19788e63-8c11-4c84-9998-c2e28b6c5908") : secret "insights-runtime-extractor-tls" not found Jul 07 17:28:18.070204 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:18.070183 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19788e63-8c11-4c84-9998-c2e28b6c5908-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:18.221502 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:18.221411 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:19.225128 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:19.225089 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:19.286804 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:19.286765 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:19.425630 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:19.425590 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:19.809793 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:19.809749 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:28:19.810042 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:19.809830 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:28:19.810303 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:19.810285 2562 scope.go:117] "RemoveContainer" containerID="68c85c129c25217e539a750a2ac3facbfbad107a75e9153885e838b2b39dc0a1" Jul 07 17:28:19.823009 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:19.822977 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:19.995172 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:19.995140 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 17:28:19.995350 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:19.995200 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" event={"ID":"8eea6af9-b0e3-4bf5-9ee8-aa45037566a3","Type":"ContainerStarted","Data":"7da87bef07936aafdf90fb892f30ab9c1cb0c6444e8f3012e60326b4f919131d"} Jul 07 17:28:19.995510 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:19.995492 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:28:20.016330 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:20.016268 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" podStartSLOduration=15.849637884 podStartE2EDuration="24.016250331s" totalImagesPullingTime="8.166612447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:27:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:20.01490177 +0000 UTC m=+55.896538298" watchObservedRunningTime="2026-07-07 17:28:20.016250331 +0000 UTC m=+55.897886859" Jul 07 17:28:20.026501 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:20.026465 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:20.148480 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:20.148443 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:20.229653 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:20.229590 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:20.329589 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:20.329560 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-nxsw4" Jul 07 17:28:20.533873 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:20.533754 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:20.634639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:20.634572 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:20.734677 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:20.734625 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:20.836211 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:20.836173 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:21.000772 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:21.000727 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:21.102157 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:21.102073 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:21.225630 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:21.225591 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:21.333154 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:21.333113 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:21.432126 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:21.432028 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:21.532757 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:21.532725 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:21.633915 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:21.633870 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:21.845286 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:21.845250 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:21.945169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:21.945133 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:22.059405 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:22.059365 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:22.417005 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:22.416945 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:22.492984 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:22.492946 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:22.677006 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:22.676903 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:23.033369 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:23.033273 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:23.233398 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:23.233363 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:23.623293 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:23.623258 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:23.808227 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:23.808194 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:23.871901 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:23.871871 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xzssf" Jul 07 17:28:23.909932 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:23.909836 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:24.019525 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:24.019483 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:24.233396 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:24.233297 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:25.418348 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:25.418314 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:25.629741 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:25.629696 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:25.818120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:25.818080 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:26.020447 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:26.020415 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:26.081847 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:26.081766 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:26.146049 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:26.146012 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:26.148522 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:26.148500 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19788e63-8c11-4c84-9998-c2e28b6c5908-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-25z9v\" (UID: \"19788e63-8c11-4c84-9998-c2e28b6c5908\") " pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:26.222215 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:26.222169 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:26.270284 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:26.270245 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8ztdc\"" Jul 07 17:28:26.277715 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:26.277689 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-25z9v" Jul 07 17:28:26.412016 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:26.411983 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-25z9v"] Jul 07 17:28:26.414800 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:26.414769 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19788e63_8c11_4c84_9998_c2e28b6c5908.slice/crio-68e885482ea18914c6c4f84235151558da567174fd5bfb6907e84adafd9a8c40 WatchSource:0}: Error finding container 68e885482ea18914c6c4f84235151558da567174fd5bfb6907e84adafd9a8c40: Status 404 returned error can't find the container with id 68e885482ea18914c6c4f84235151558da567174fd5bfb6907e84adafd9a8c40 Jul 07 17:28:26.624512 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:26.624425 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:27.021860 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:27.021772 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-25z9v" event={"ID":"19788e63-8c11-4c84-9998-c2e28b6c5908","Type":"ContainerStarted","Data":"68e885482ea18914c6c4f84235151558da567174fd5bfb6907e84adafd9a8c40"} Jul 07 17:28:30.285536 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:30.285448 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs\") pod \"network-metrics-daemon-7mtf5\" (UID: \"3e184935-aec2-42e9-8944-5199f4d6c1b9\") " pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:28:30.288831 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:30.288809 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 17:28:30.298713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:30.298674 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e184935-aec2-42e9-8944-5199f4d6c1b9-metrics-certs\") pod \"network-metrics-daemon-7mtf5\" (UID: \"3e184935-aec2-42e9-8944-5199f4d6c1b9\") " pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:28:30.386716 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:30.386675 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret\") pod \"global-pull-secret-syncer-5nf29\" (UID: \"658cefc3-6811-4b18-b3a2-3af9ac357158\") " pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:28:30.386716 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:30.386723 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd6b6\" (UniqueName: \"kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6\") pod \"network-check-target-47448\" (UID: \"688bff8d-e98a-4a83-a526-18eab4e56743\") " pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:28:30.389346 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:30.389324 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd6b6\" (UniqueName: \"kubernetes.io/projected/688bff8d-e98a-4a83-a526-18eab4e56743-kube-api-access-sd6b6\") pod \"network-check-target-47448\" (UID: \"688bff8d-e98a-4a83-a526-18eab4e56743\") " pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:28:30.389876 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:30.389854 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 17:28:30.400004 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:30.399953 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658cefc3-6811-4b18-b3a2-3af9ac357158-original-pull-secret\") pod \"global-pull-secret-syncer-5nf29\" (UID: \"658cefc3-6811-4b18-b3a2-3af9ac357158\") " pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:28:30.544694 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:30.544619 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hxlp2\"" Jul 07 17:28:30.547090 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:30.547070 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nf29" Jul 07 17:28:30.552424 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:30.552389 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:28:30.554181 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:30.554164 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v8q6k\"" Jul 07 17:28:30.561538 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:30.561513 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtf5" Jul 07 17:28:30.778987 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:30.778932 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7mtf5"] Jul 07 17:28:30.783713 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:30.783684 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e184935_aec2_42e9_8944_5199f4d6c1b9.slice/crio-79124a55e9112ce507404a4f5e1ffdf9abef17a464a1a5c060315d3c544c4daf WatchSource:0}: Error finding container 79124a55e9112ce507404a4f5e1ffdf9abef17a464a1a5c060315d3c544c4daf: Status 404 returned error can't find the container with id 79124a55e9112ce507404a4f5e1ffdf9abef17a464a1a5c060315d3c544c4daf Jul 07 17:28:30.796602 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:30.796532 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5nf29"] Jul 07 17:28:30.800490 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:30.800461 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod658cefc3_6811_4b18_b3a2_3af9ac357158.slice/crio-43924ae8cd1a71fec93af52df8fd5c935cda8124d1bc3986a544ababd71c8390 WatchSource:0}: Error finding container 43924ae8cd1a71fec93af52df8fd5c935cda8124d1bc3986a544ababd71c8390: Status 404 returned error can't find the container with id 43924ae8cd1a71fec93af52df8fd5c935cda8124d1bc3986a544ababd71c8390 Jul 07 17:28:30.810709 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:30.810682 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-47448"] Jul 07 17:28:30.813270 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:30.813244 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod688bff8d_e98a_4a83_a526_18eab4e56743.slice/crio-e44f542108616cfad0815fe60a6a3d171b055ef41615ce5faf5cde75ab9c7507 WatchSource:0}: Error finding container e44f542108616cfad0815fe60a6a3d171b055ef41615ce5faf5cde75ab9c7507: Status 404 returned error can't find the container with id e44f542108616cfad0815fe60a6a3d171b055ef41615ce5faf5cde75ab9c7507 Jul 07 17:28:31.040044 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.040005 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-25z9v" event={"ID":"19788e63-8c11-4c84-9998-c2e28b6c5908","Type":"ContainerStarted","Data":"68ebfeda12fe69e5988c377c328ca693b84d102bfab465aaac2b3ab4c5d34808"} Jul 07 17:28:31.040202 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.040063 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-25z9v" event={"ID":"19788e63-8c11-4c84-9998-c2e28b6c5908","Type":"ContainerStarted","Data":"9bd2eb332d6a1ed14372fd8ed4cffed3e966c898a39c2727d455c129fbdc6c82"} Jul 07 17:28:31.041270 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.041242 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-47448" event={"ID":"688bff8d-e98a-4a83-a526-18eab4e56743","Type":"ContainerStarted","Data":"14ac4b69513095388fab9ce3fdf2f1b65c412ed36c926cedca6331d3b555df6d"} Jul 07 17:28:31.041414 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.041287 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-47448" event={"ID":"688bff8d-e98a-4a83-a526-18eab4e56743","Type":"ContainerStarted","Data":"e44f542108616cfad0815fe60a6a3d171b055ef41615ce5faf5cde75ab9c7507"} Jul 07 17:28:31.041414 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.041330 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:28:31.042326 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.042305 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7mtf5" event={"ID":"3e184935-aec2-42e9-8944-5199f4d6c1b9","Type":"ContainerStarted","Data":"79124a55e9112ce507404a4f5e1ffdf9abef17a464a1a5c060315d3c544c4daf"} Jul 07 17:28:31.043280 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.043260 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5nf29" event={"ID":"658cefc3-6811-4b18-b3a2-3af9ac357158","Type":"ContainerStarted","Data":"43924ae8cd1a71fec93af52df8fd5c935cda8124d1bc3986a544ababd71c8390"} Jul 07 17:28:31.063080 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.063030 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-47448" podStartSLOduration=67.063012738 podStartE2EDuration="1m7.063012738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:27:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:31.062019782 +0000 UTC m=+66.943656309" watchObservedRunningTime="2026-07-07 17:28:31.063012738 +0000 UTC m=+66.944649242" Jul 07 17:28:31.396480 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.396386 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:28:31.396480 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.396450 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:28:31.397010 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.396485 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:31.397010 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.396518 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52xr5\" (UID: \"2308b91e-a103-451e-b06f-fc5bb0b50c97\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:28:31.397010 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.396569 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:31.397010 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.396601 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l42k\" (UID: \"8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" Jul 07 17:28:31.397955 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.397917 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6c8843e-708e-4e0d-a538-e35c93a4beec-service-ca-bundle\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:31.403007 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.402951 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2308b91e-a103-451e-b06f-fc5bb0b50c97-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-52xr5\" (UID: \"2308b91e-a103-451e-b06f-fc5bb0b50c97\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:28:31.403007 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.403005 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l42k\" (UID: \"8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" Jul 07 17:28:31.403248 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.402952 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6c8843e-708e-4e0d-a538-e35c93a4beec-metrics-certs\") pod \"router-default-7fbfb58cb7-68cb8\" (UID: \"b6c8843e-708e-4e0d-a538-e35c93a4beec\") " pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:31.404744 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.404717 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls\") pod \"image-registry-7fc965b778-q5jc7\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:28:31.406170 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.406105 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dcbb03e-a464-408b-a012-8fc6a386ef16-metrics-tls\") pod \"dns-default-n4h5f\" (UID: \"9dcbb03e-a464-408b-a012-8fc6a386ef16\") " pod="openshift-dns/dns-default-n4h5f" Jul 07 17:28:31.541228 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.541189 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lzxpj\"" Jul 07 17:28:31.548704 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.548666 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:28:31.557139 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.557104 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xm8zg\"" Jul 07 17:28:31.565371 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.565317 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:31.570728 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.570469 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-tg4q6\"" Jul 07 17:28:31.579328 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.579045 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" Jul 07 17:28:31.604552 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.604516 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jpt5n\"" Jul 07 17:28:31.611717 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.611675 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" Jul 07 17:28:31.635899 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.635394 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-78k7c\"" Jul 07 17:28:31.642336 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.641734 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n4h5f" Jul 07 17:28:31.794047 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.792625 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fc965b778-q5jc7"] Jul 07 17:28:31.876335 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.876200 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k"] Jul 07 17:28:31.920772 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:31.920693 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n4h5f"] Jul 07 17:28:32.052149 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:32.052049 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" event={"ID":"e4a2ea0f-ff58-4688-b947-46d4352afc62","Type":"ContainerStarted","Data":"c40825e01839ca48cbfe4a731f0083d3fb09b22be4d91385df901ae60c9ec8a0"} Jul 07 17:28:32.052149 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:32.052092 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" event={"ID":"e4a2ea0f-ff58-4688-b947-46d4352afc62","Type":"ContainerStarted","Data":"02254ab35d999962f04d27dbcbcf0d90a392496be7640d125ff3eaf52681ee24"} Jul 07 17:28:32.052149 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:32.052121 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:28:32.064909 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:32.064886 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5"] Jul 07 17:28:32.071576 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:32.071545 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7fbfb58cb7-68cb8"] Jul 07 17:28:32.111610 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:32.111543 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" podStartSLOduration=67.111521611 podStartE2EDuration="1m7.111521611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:27:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:32.110862915 +0000 UTC m=+67.992499445" watchObservedRunningTime="2026-07-07 17:28:32.111521611 +0000 UTC m=+67.993158139" Jul 07 17:28:33.058543 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:33.058499 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" event={"ID":"2308b91e-a103-451e-b06f-fc5bb0b50c97","Type":"ContainerStarted","Data":"59eb617b3cb44624b1bfaeee8dd29767a610f187f081d6242620b60206e3813d"} Jul 07 17:28:33.060385 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:33.060347 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n4h5f" event={"ID":"9dcbb03e-a464-408b-a012-8fc6a386ef16","Type":"ContainerStarted","Data":"49599f7517a73bb9045292362263fe4936038b740ba6775024b9c17536982c99"} Jul 07 17:28:33.062035 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:33.062004 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" event={"ID":"b6c8843e-708e-4e0d-a538-e35c93a4beec","Type":"ContainerStarted","Data":"d93f5876716a6720a59abc000bc4cf697c577fe482e306d7696ae8a9f6673b88"} Jul 07 17:28:33.062154 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:33.062057 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" event={"ID":"b6c8843e-708e-4e0d-a538-e35c93a4beec","Type":"ContainerStarted","Data":"2f1f2ab7a5505945dc89a5ff3af8e81e28c3769190f2796c4aa2e535e3dac0d4"} Jul 07 17:28:33.065460 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:33.065428 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7mtf5" event={"ID":"3e184935-aec2-42e9-8944-5199f4d6c1b9","Type":"ContainerStarted","Data":"fa46fd5c64ddc348da361af0d35b84806e3af9892c624f300450a76f5bfec0c4"} Jul 07 17:28:33.065559 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:33.065491 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7mtf5" event={"ID":"3e184935-aec2-42e9-8944-5199f4d6c1b9","Type":"ContainerStarted","Data":"2c48587cee23c3cf5163bcb1cc8147a53898f6b6a74f20b04cf9ebc8fad122e9"} Jul 07 17:28:33.068556 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:33.068528 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" event={"ID":"8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b","Type":"ContainerStarted","Data":"e47ff3aa39d5dcd34b4d5b1272d0bb1695b0541f8681769c91242b20c053d567"} Jul 07 17:28:33.089396 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:33.089324 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" podStartSLOduration=47.089302007 podStartE2EDuration="47.089302007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:27:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:33.087521584 +0000 UTC m=+68.969158124" watchObservedRunningTime="2026-07-07 17:28:33.089302007 +0000 UTC m=+68.970938537" Jul 07 17:28:33.107940 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:33.107874 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7mtf5" podStartSLOduration=67.633856256 podStartE2EDuration="1m9.107849774s" totalImagesPullingTime="1.473993518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:33.106655477 +0000 UTC m=+68.988292013" watchObservedRunningTime="2026-07-07 17:28:33.107849774 +0000 UTC m=+68.989486303" Jul 07 17:28:33.565639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:33.565557 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:33.569407 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:33.569153 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:33.837598 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:33.837511 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:33.937467 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:33.937427 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:34.073323 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:34.073289 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:34.075154 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:34.074920 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7fbfb58cb7-68cb8" Jul 07 17:28:35.504165 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.504127 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4"] Jul 07 17:28:35.509738 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.509708 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g"] Jul 07 17:28:35.509898 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.509846 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4" Jul 07 17:28:35.513018 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.512991 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 17:28:35.513377 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.513198 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 17:28:35.513377 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.513204 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 07 17:28:35.514244 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.514220 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g" Jul 07 17:28:35.516617 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.516114 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 17:28:35.517232 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.517209 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-wdz2d\"" Jul 07 17:28:35.529123 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.523271 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 07 17:28:35.532954 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.532847 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g"] Jul 07 17:28:35.535593 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.534677 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4"] Jul 07 17:28:35.537240 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.536951 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq"] Jul 07 17:28:35.545078 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.544299 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.548101 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.548077 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq"] Jul 07 17:28:35.548899 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.548877 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 07 17:28:35.549364 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.549316 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 07 17:28:35.550010 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.549615 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 07 17:28:35.550010 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.549675 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 07 17:28:35.617574 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.617475 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-mpf9p"] Jul 07 17:28:35.624403 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.624374 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-45d77"] Jul 07 17:28:35.624587 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.624497 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-mpf9p" Jul 07 17:28:35.628020 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.627739 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 17:28:35.628020 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.627740 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 17:28:35.628020 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.627867 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-ntwhq\"" Jul 07 17:28:35.631372 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.631353 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-45d77" Jul 07 17:28:35.634875 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.634797 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 07 17:28:35.635679 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.635156 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-rm9wx\"" Jul 07 17:28:35.635679 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.635183 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 07 17:28:35.639120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.637560 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-45d77"] Jul 07 17:28:35.639120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.637836 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pp9w\" (UniqueName: \"kubernetes.io/projected/f7568f8e-0e19-4255-9540-05850c6a0821-kube-api-access-9pp9w\") pod \"klusterlet-addon-workmgr-b44f9b595-bjhj4\" (UID: \"f7568f8e-0e19-4255-9540-05850c6a0821\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4" Jul 07 17:28:35.639120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.637872 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5shxq\" (UniqueName: \"kubernetes.io/projected/86565be3-5425-49c8-8d82-c246d33c50a4-kube-api-access-5shxq\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.639120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.637949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7568f8e-0e19-4255-9540-05850c6a0821-tmp\") pod \"klusterlet-addon-workmgr-b44f9b595-bjhj4\" (UID: \"f7568f8e-0e19-4255-9540-05850c6a0821\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4" Jul 07 17:28:35.639120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.638000 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/86565be3-5425-49c8-8d82-c246d33c50a4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.639120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.638031 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86565be3-5425-49c8-8d82-c246d33c50a4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.639120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.638058 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/419724d5-cabf-4577-8d32-c1476cc8ea03-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g\" (UID: \"419724d5-cabf-4577-8d32-c1476cc8ea03\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g" Jul 07 17:28:35.639120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.638163 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f7568f8e-0e19-4255-9540-05850c6a0821-klusterlet-config\") pod \"klusterlet-addon-workmgr-b44f9b595-bjhj4\" (UID: \"f7568f8e-0e19-4255-9540-05850c6a0821\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4" Jul 07 17:28:35.639120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.638195 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/86565be3-5425-49c8-8d82-c246d33c50a4-hub\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.639120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.638229 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb4xv\" (UniqueName: \"kubernetes.io/projected/419724d5-cabf-4577-8d32-c1476cc8ea03-kube-api-access-tb4xv\") pod \"managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g\" (UID: \"419724d5-cabf-4577-8d32-c1476cc8ea03\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g" Jul 07 17:28:35.639120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.638256 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/86565be3-5425-49c8-8d82-c246d33c50a4-ca\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.639120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.638295 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/86565be3-5425-49c8-8d82-c246d33c50a4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.639120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.639073 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-mpf9p"] Jul 07 17:28:35.643630 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.643465 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:35.739613 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.739575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5shxq\" (UniqueName: \"kubernetes.io/projected/86565be3-5425-49c8-8d82-c246d33c50a4-kube-api-access-5shxq\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.739844 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.739623 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7568f8e-0e19-4255-9540-05850c6a0821-tmp\") pod \"klusterlet-addon-workmgr-b44f9b595-bjhj4\" (UID: \"f7568f8e-0e19-4255-9540-05850c6a0821\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4" Jul 07 17:28:35.739844 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.739650 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/86565be3-5425-49c8-8d82-c246d33c50a4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.739844 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.739690 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86565be3-5425-49c8-8d82-c246d33c50a4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.739844 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.739724 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/419724d5-cabf-4577-8d32-c1476cc8ea03-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g\" (UID: \"419724d5-cabf-4577-8d32-c1476cc8ea03\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g" Jul 07 17:28:35.739844 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.739781 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88acb68c-7353-4a80-a631-e7fe9713cfb4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-45d77\" (UID: \"88acb68c-7353-4a80-a631-e7fe9713cfb4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-45d77" Jul 07 17:28:35.739844 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.739811 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/88acb68c-7353-4a80-a631-e7fe9713cfb4-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-45d77\" (UID: \"88acb68c-7353-4a80-a631-e7fe9713cfb4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-45d77" Jul 07 17:28:35.739844 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.739842 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f7568f8e-0e19-4255-9540-05850c6a0821-klusterlet-config\") pod \"klusterlet-addon-workmgr-b44f9b595-bjhj4\" (UID: \"f7568f8e-0e19-4255-9540-05850c6a0821\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4" Jul 07 17:28:35.740319 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.739870 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/86565be3-5425-49c8-8d82-c246d33c50a4-hub\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.740319 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.739911 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb4xv\" (UniqueName: \"kubernetes.io/projected/419724d5-cabf-4577-8d32-c1476cc8ea03-kube-api-access-tb4xv\") pod \"managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g\" (UID: \"419724d5-cabf-4577-8d32-c1476cc8ea03\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g" Jul 07 17:28:35.740319 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.739952 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/86565be3-5425-49c8-8d82-c246d33c50a4-ca\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.740319 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.740004 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dl7q\" (UniqueName: \"kubernetes.io/projected/8a560068-6fa5-4afa-96d5-06792a244208-kube-api-access-6dl7q\") pod \"downloads-685bfb5f64-mpf9p\" (UID: \"8a560068-6fa5-4afa-96d5-06792a244208\") " pod="openshift-console/downloads-685bfb5f64-mpf9p" Jul 07 17:28:35.740319 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.740053 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/86565be3-5425-49c8-8d82-c246d33c50a4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.740319 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.740097 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pp9w\" (UniqueName: \"kubernetes.io/projected/f7568f8e-0e19-4255-9540-05850c6a0821-kube-api-access-9pp9w\") pod \"klusterlet-addon-workmgr-b44f9b595-bjhj4\" (UID: \"f7568f8e-0e19-4255-9540-05850c6a0821\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4" Jul 07 17:28:35.740319 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.740106 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7568f8e-0e19-4255-9540-05850c6a0821-tmp\") pod \"klusterlet-addon-workmgr-b44f9b595-bjhj4\" (UID: \"f7568f8e-0e19-4255-9540-05850c6a0821\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4" Jul 07 17:28:35.741369 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.741340 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/86565be3-5425-49c8-8d82-c246d33c50a4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.742762 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.742723 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:35.743550 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.743498 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/419724d5-cabf-4577-8d32-c1476cc8ea03-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g\" (UID: \"419724d5-cabf-4577-8d32-c1476cc8ea03\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g" Jul 07 17:28:35.743920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.743895 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/86565be3-5425-49c8-8d82-c246d33c50a4-hub\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.744166 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.744142 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/86565be3-5425-49c8-8d82-c246d33c50a4-ca\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.744258 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.744162 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/86565be3-5425-49c8-8d82-c246d33c50a4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.744710 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.744669 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f7568f8e-0e19-4255-9540-05850c6a0821-klusterlet-config\") pod \"klusterlet-addon-workmgr-b44f9b595-bjhj4\" (UID: \"f7568f8e-0e19-4255-9540-05850c6a0821\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4" Jul 07 17:28:35.745334 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.745311 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86565be3-5425-49c8-8d82-c246d33c50a4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.755772 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.755742 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pp9w\" (UniqueName: \"kubernetes.io/projected/f7568f8e-0e19-4255-9540-05850c6a0821-kube-api-access-9pp9w\") pod \"klusterlet-addon-workmgr-b44f9b595-bjhj4\" (UID: \"f7568f8e-0e19-4255-9540-05850c6a0821\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4" Jul 07 17:28:35.759830 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.759798 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5shxq\" (UniqueName: \"kubernetes.io/projected/86565be3-5425-49c8-8d82-c246d33c50a4-kube-api-access-5shxq\") pod \"cluster-proxy-proxy-agent-75d6d65cc7-6g8jq\" (UID: \"86565be3-5425-49c8-8d82-c246d33c50a4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.766998 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.766940 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb4xv\" (UniqueName: \"kubernetes.io/projected/419724d5-cabf-4577-8d32-c1476cc8ea03-kube-api-access-tb4xv\") pod \"managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g\" (UID: \"419724d5-cabf-4577-8d32-c1476cc8ea03\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g" Jul 07 17:28:35.833894 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.833860 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4" Jul 07 17:28:35.841212 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.841168 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88acb68c-7353-4a80-a631-e7fe9713cfb4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-45d77\" (UID: \"88acb68c-7353-4a80-a631-e7fe9713cfb4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-45d77" Jul 07 17:28:35.841381 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.841223 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/88acb68c-7353-4a80-a631-e7fe9713cfb4-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-45d77\" (UID: \"88acb68c-7353-4a80-a631-e7fe9713cfb4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-45d77" Jul 07 17:28:35.841381 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.841259 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dl7q\" (UniqueName: \"kubernetes.io/projected/8a560068-6fa5-4afa-96d5-06792a244208-kube-api-access-6dl7q\") pod \"downloads-685bfb5f64-mpf9p\" (UID: \"8a560068-6fa5-4afa-96d5-06792a244208\") " pod="openshift-console/downloads-685bfb5f64-mpf9p" Jul 07 17:28:35.842145 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.842042 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/88acb68c-7353-4a80-a631-e7fe9713cfb4-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-45d77\" (UID: \"88acb68c-7353-4a80-a631-e7fe9713cfb4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-45d77" Jul 07 17:28:35.843949 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.843926 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88acb68c-7353-4a80-a631-e7fe9713cfb4-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-45d77\" (UID: \"88acb68c-7353-4a80-a631-e7fe9713cfb4\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-45d77" Jul 07 17:28:35.853834 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.853807 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g" Jul 07 17:28:35.857116 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.857090 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dl7q\" (UniqueName: \"kubernetes.io/projected/8a560068-6fa5-4afa-96d5-06792a244208-kube-api-access-6dl7q\") pod \"downloads-685bfb5f64-mpf9p\" (UID: \"8a560068-6fa5-4afa-96d5-06792a244208\") " pod="openshift-console/downloads-685bfb5f64-mpf9p" Jul 07 17:28:35.863865 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.863842 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" Jul 07 17:28:35.932368 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.932278 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:35.954480 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.954436 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-mpf9p" Jul 07 17:28:35.962129 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:35.962094 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-45d77" Jul 07 17:28:36.037313 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:36.037263 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:36.135269 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:36.135164 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:36.202181 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:36.202139 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:36.582539 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:36.582437 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:36.643149 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:36.643107 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:37.104255 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:37.102880 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-25z9v" event={"ID":"19788e63-8c11-4c84-9998-c2e28b6c5908","Type":"ContainerStarted","Data":"ba599e97e5f116a26b78704ce8041e28625bd9112f8fcd8dda7ce6aa05433b33"} Jul 07 17:28:37.132056 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:37.131612 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-25z9v" podStartSLOduration=16.770322211 podStartE2EDuration="27.13158794s" totalImagesPullingTime="10.361265729s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:28:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:37.129414708 +0000 UTC m=+73.011051259" watchObservedRunningTime="2026-07-07 17:28:37.13158794 +0000 UTC m=+73.013224467" Jul 07 17:28:37.155114 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:37.155086 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-mpf9p"] Jul 07 17:28:37.163511 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:37.163097 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g"] Jul 07 17:28:37.164560 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:37.164491 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4"] Jul 07 17:28:37.249198 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:37.249168 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a560068_6fa5_4afa_96d5_06792a244208.slice/crio-09e114fa14f3096d92e35860ce9bf9a05e184dd9946838c77b833870b7354f6e WatchSource:0}: Error finding container 09e114fa14f3096d92e35860ce9bf9a05e184dd9946838c77b833870b7354f6e: Status 404 returned error can't find the container with id 09e114fa14f3096d92e35860ce9bf9a05e184dd9946838c77b833870b7354f6e Jul 07 17:28:37.250936 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:37.250547 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod419724d5_cabf_4577_8d32_c1476cc8ea03.slice/crio-b927e31c9cf4d94646eb69a427de9fae798a1d2e88ca7b0dfe74c3988efb9133 WatchSource:0}: Error finding container b927e31c9cf4d94646eb69a427de9fae798a1d2e88ca7b0dfe74c3988efb9133: Status 404 returned error can't find the container with id b927e31c9cf4d94646eb69a427de9fae798a1d2e88ca7b0dfe74c3988efb9133 Jul 07 17:28:37.250936 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:37.250756 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7568f8e_0e19_4255_9540_05850c6a0821.slice/crio-2b6212994f9ae7b15205fdd494f0c638b63e35820e6b5a589a4ffc1992500db1 WatchSource:0}: Error finding container 2b6212994f9ae7b15205fdd494f0c638b63e35820e6b5a589a4ffc1992500db1: Status 404 returned error can't find the container with id 2b6212994f9ae7b15205fdd494f0c638b63e35820e6b5a589a4ffc1992500db1 Jul 07 17:28:37.376066 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:37.375942 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:37.401392 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:37.401212 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-45d77"] Jul 07 17:28:37.405130 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:37.404874 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq"] Jul 07 17:28:37.466349 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:37.466312 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:37.609425 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:37.609387 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:37.707778 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:37.707691 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:37.821293 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:37.821252 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:37.876882 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:37.876852 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86565be3_5425_49c8_8d82_c246d33c50a4.slice/crio-9a3193c0261348229d5a965c15e3fbe7edbe9e8c26f8310c96b81526b2b0ca9d WatchSource:0}: Error finding container 9a3193c0261348229d5a965c15e3fbe7edbe9e8c26f8310c96b81526b2b0ca9d: Status 404 returned error can't find the container with id 9a3193c0261348229d5a965c15e3fbe7edbe9e8c26f8310c96b81526b2b0ca9d Jul 07 17:28:37.877584 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:37.877563 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88acb68c_7353_4a80_a631_e7fe9713cfb4.slice/crio-b3caf375d089e9308e04622263a401e38cd36513807ce4b4be036459cd477979 WatchSource:0}: Error finding container b3caf375d089e9308e04622263a401e38cd36513807ce4b4be036459cd477979: Status 404 returned error can't find the container with id b3caf375d089e9308e04622263a401e38cd36513807ce4b4be036459cd477979 Jul 07 17:28:37.908586 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:37.908541 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:38.108790 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:38.108749 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" event={"ID":"2308b91e-a103-451e-b06f-fc5bb0b50c97","Type":"ContainerStarted","Data":"c47050c5ea87c87e587fac250c6a410568b73519051790f1340e56b875a739ed"} Jul 07 17:28:38.110811 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:38.110776 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n4h5f" event={"ID":"9dcbb03e-a464-408b-a012-8fc6a386ef16","Type":"ContainerStarted","Data":"6951d2ca68465c0c92af52229a10ea5dd7feb17ffd37f2245552f09c0d078466"} Jul 07 17:28:38.110993 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:38.110839 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-n4h5f" Jul 07 17:28:38.110993 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:38.110865 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n4h5f" event={"ID":"9dcbb03e-a464-408b-a012-8fc6a386ef16","Type":"ContainerStarted","Data":"435b546c7f800f8c7611db1f51315bf167a55a79c24f22a582b26867bd2cef2f"} Jul 07 17:28:38.111992 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:38.111947 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4" event={"ID":"f7568f8e-0e19-4255-9540-05850c6a0821","Type":"ContainerStarted","Data":"2b6212994f9ae7b15205fdd494f0c638b63e35820e6b5a589a4ffc1992500db1"} Jul 07 17:28:38.113144 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:38.113110 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g" event={"ID":"419724d5-cabf-4577-8d32-c1476cc8ea03","Type":"ContainerStarted","Data":"b927e31c9cf4d94646eb69a427de9fae798a1d2e88ca7b0dfe74c3988efb9133"} Jul 07 17:28:38.114257 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:38.114218 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" event={"ID":"86565be3-5425-49c8-8d82-c246d33c50a4","Type":"ContainerStarted","Data":"9a3193c0261348229d5a965c15e3fbe7edbe9e8c26f8310c96b81526b2b0ca9d"} Jul 07 17:28:38.115762 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:38.115457 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-mpf9p" event={"ID":"8a560068-6fa5-4afa-96d5-06792a244208","Type":"ContainerStarted","Data":"09e114fa14f3096d92e35860ce9bf9a05e184dd9946838c77b833870b7354f6e"} Jul 07 17:28:38.116942 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:38.116917 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-45d77" event={"ID":"88acb68c-7353-4a80-a631-e7fe9713cfb4","Type":"ContainerStarted","Data":"b3caf375d089e9308e04622263a401e38cd36513807ce4b4be036459cd477979"} Jul 07 17:28:38.118669 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:38.118496 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5nf29" event={"ID":"658cefc3-6811-4b18-b3a2-3af9ac357158","Type":"ContainerStarted","Data":"fd1f6e73cd89a2c00ac3d56ebe93b3878e300c8c0aa0c425379302daf189f7fc"} Jul 07 17:28:38.131587 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:38.131534 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-52xr5" podStartSLOduration=47.405536091 podStartE2EDuration="52.131516128s" totalImagesPullingTime="4.725980037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:38.130390309 +0000 UTC m=+74.012026836" watchObservedRunningTime="2026-07-07 17:28:38.131516128 +0000 UTC m=+74.013152648" Jul 07 17:28:38.149675 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:38.149608 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-n4h5f" podStartSLOduration=34.431317133 podStartE2EDuration="39.149585287s" totalImagesPullingTime="4.718268154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:38.149105948 +0000 UTC m=+74.030742479" watchObservedRunningTime="2026-07-07 17:28:38.149585287 +0000 UTC m=+74.031221816" Jul 07 17:28:38.167342 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:38.167281 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5nf29" podStartSLOduration=66.091166383 podStartE2EDuration="1m13.167257011s" totalImagesPullingTime="7.076090628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:27:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:38.165532794 +0000 UTC m=+74.047169329" watchObservedRunningTime="2026-07-07 17:28:38.167257011 +0000 UTC m=+74.048893518" Jul 07 17:28:38.807622 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:38.807519 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:38.873281 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:38.873239 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:39.132066 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:39.131151 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" event={"ID":"8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b","Type":"ContainerStarted","Data":"31e9d0c7a0fc845ec768fbfd307c2fec7fccfe4afe2178e0f1dea858d1d5430e"} Jul 07 17:28:39.132066 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:39.131210 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" event={"ID":"8c5402ff-f3ec-47c6-ba38-a83bfbaa3d5b","Type":"ContainerStarted","Data":"8671fb4883d8643c2038304b03d0c9d39b466de39ae9d1dc04c61d34c5669623"} Jul 07 17:28:40.662907 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:40.662833 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:40.762514 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:40.762474 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:40.983891 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:40.983804 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:41.083716 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:41.083680 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:41.253674 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:41.253494 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:41.354565 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:41.354525 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:41.765984 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:41.765939 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:41.830908 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:41.830871 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:42.040597 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:42.040500 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:42.112742 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:42.112709 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:42.340548 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:42.340505 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:42.446863 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:42.446827 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:42.517408 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:42.517322 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:43.145424 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:43.145381 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:43.159166 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:43.159136 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-45d77" event={"ID":"88acb68c-7353-4a80-a631-e7fe9713cfb4","Type":"ContainerStarted","Data":"0aac483d9867cf507b610cfdcf254ffd0ac76e9fee37f48c44d71be2494cbc89"} Jul 07 17:28:43.182737 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:43.182669 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-45d77" podStartSLOduration=3.090846105 podStartE2EDuration="8.182648564s" totalImagesPullingTime="5.091802459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:28:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:43.18215362 +0000 UTC m=+79.063790148" watchObservedRunningTime="2026-07-07 17:28:43.182648564 +0000 UTC m=+79.064285091" Jul 07 17:28:43.183822 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:43.183770 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l42k" podStartSLOduration=41.178202615000004 podStartE2EDuration="47.183750947s" totalImagesPullingTime="6.005548332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:27:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:39.156156021 +0000 UTC m=+75.037792561" watchObservedRunningTime="2026-07-07 17:28:43.183750947 +0000 UTC m=+79.065387477" Jul 07 17:28:43.247093 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:43.246942 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:43.562683 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:43.562633 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:43.682770 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:43.682711 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:44.346619 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:44.346562 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:44.780075 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:44.779937 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:44.870452 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:44.870420 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:44.939741 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:44.939709 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:45.176981 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:45.176922 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4" event={"ID":"f7568f8e-0e19-4255-9540-05850c6a0821","Type":"ContainerStarted","Data":"c5e398dc091a325d0be73d6df360d1ac38a92fa12e022bf95c6875830d18143d"} Jul 07 17:28:45.177175 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:45.177077 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4" Jul 07 17:28:45.179312 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:45.179292 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4" Jul 07 17:28:45.196508 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:45.196442 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b44f9b595-bjhj4" podStartSLOduration=2.630356101 podStartE2EDuration="10.196419833s" totalImagesPullingTime="7.566063732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:28:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:45.195582905 +0000 UTC m=+81.077219434" watchObservedRunningTime="2026-07-07 17:28:45.196419833 +0000 UTC m=+81.078056364" Jul 07 17:28:45.394790 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:45.394760 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:45.492103 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:45.492062 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:46.182804 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:46.182767 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" event={"ID":"86565be3-5425-49c8-8d82-c246d33c50a4","Type":"ContainerStarted","Data":"f0584cb3c05e7dcc8caa256e2891df05a70087d005f71c3598426f809381307e"} Jul 07 17:28:46.182804 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:46.182812 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" event={"ID":"86565be3-5425-49c8-8d82-c246d33c50a4","Type":"ContainerStarted","Data":"77d1e07091d2c97e83a9de0196afdc33bb2e280a357b3de547ef385c7b7e9bfe"} Jul 07 17:28:46.183101 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:46.182825 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" event={"ID":"86565be3-5425-49c8-8d82-c246d33c50a4","Type":"ContainerStarted","Data":"880d9ee42749be342c21bd2c9390c8fd4ec8c88de9ed3c96bc1afd5916892045"} Jul 07 17:28:46.207283 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:46.207214 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-75d6d65cc7-6g8jq" podStartSLOduration=3.879213451 podStartE2EDuration="11.207190264s" totalImagesPullingTime="7.327976813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:28:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:46.20499982 +0000 UTC m=+82.086636348" watchObservedRunningTime="2026-07-07 17:28:46.207190264 +0000 UTC m=+82.088826793" Jul 07 17:28:46.983191 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:46.983150 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:47.056739 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:47.056701 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:48.133310 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:48.133274 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-n4h5f" Jul 07 17:28:48.962864 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:48.962827 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:50.401996 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:50.401942 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:50.472196 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:50.472157 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:51.034367 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.034320 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:51.279706 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.279654 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:51.321888 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.321856 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dncvg"] Jul 07 17:28:51.325265 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.325237 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.328532 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.328510 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 17:28:51.329369 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.329090 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 17:28:51.329369 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.329189 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 17:28:51.329606 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.329591 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qgfqj\"" Jul 07 17:28:51.329883 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.329857 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 17:28:51.375736 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.375692 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:51.471990 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.471719 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:51.493985 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.493443 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.493985 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.493526 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68e5e3ca-2621-475d-97e2-b0a891b4abd9-metrics-client-ca\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.493985 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.493856 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-accelerators-collector-config\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.494257 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.493937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-tls\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.494257 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.494091 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-wtmp\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.494257 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.494147 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-textfile\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.494257 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.494237 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68e5e3ca-2621-475d-97e2-b0a891b4abd9-root\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.494445 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.494281 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwdqm\" (UniqueName: \"kubernetes.io/projected/68e5e3ca-2621-475d-97e2-b0a891b4abd9-kube-api-access-hwdqm\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.494445 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.494306 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68e5e3ca-2621-475d-97e2-b0a891b4abd9-sys\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.557476 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.557435 2562 patch_prober.go:28] interesting pod/image-registry-7fc965b778-q5jc7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 07 17:28:51.557640 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.557515 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" podUID="e4a2ea0f-ff58-4688-b947-46d4352afc62" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 17:28:51.581662 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.581507 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:51.595264 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.595234 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68e5e3ca-2621-475d-97e2-b0a891b4abd9-metrics-client-ca\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.595467 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.595280 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-accelerators-collector-config\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.595467 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.595322 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-tls\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.595467 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.595351 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-wtmp\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.595467 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.595375 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-textfile\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.595467 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.595398 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68e5e3ca-2621-475d-97e2-b0a891b4abd9-root\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.595467 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.595418 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwdqm\" (UniqueName: \"kubernetes.io/projected/68e5e3ca-2621-475d-97e2-b0a891b4abd9-kube-api-access-hwdqm\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.595467 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.595440 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68e5e3ca-2621-475d-97e2-b0a891b4abd9-sys\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.595793 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.595498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.596670 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.596050 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68e5e3ca-2621-475d-97e2-b0a891b4abd9-root\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.596670 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.596060 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-wtmp\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.596670 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.596338 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-textfile\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.596670 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:51.596426 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 17:28:51.596670 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:51.596472 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-tls podName:68e5e3ca-2621-475d-97e2-b0a891b4abd9 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:52.096456831 +0000 UTC m=+87.978093335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-tls") pod "node-exporter-dncvg" (UID: "68e5e3ca-2621-475d-97e2-b0a891b4abd9") : secret "node-exporter-tls" not found Jul 07 17:28:51.596670 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.596631 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68e5e3ca-2621-475d-97e2-b0a891b4abd9-sys\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.597081 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.596857 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-accelerators-collector-config\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.597210 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.597190 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68e5e3ca-2621-475d-97e2-b0a891b4abd9-metrics-client-ca\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.598103 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.598061 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.615208 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.615183 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwdqm\" (UniqueName: \"kubernetes.io/projected/68e5e3ca-2621-475d-97e2-b0a891b4abd9-kube-api-access-hwdqm\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:51.711349 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.710053 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:51.793197 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.793164 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:51.926851 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.926773 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:51.998870 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:51.997893 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:52.103976 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:52.103941 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-tls\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:52.104182 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:52.104142 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 17:28:52.104299 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:28:52.104244 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-tls podName:68e5e3ca-2621-475d-97e2-b0a891b4abd9 nodeName:}" failed. No retries permitted until 2026-07-07 17:28:53.104221754 +0000 UTC m=+88.985858278 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-tls") pod "node-exporter-dncvg" (UID: "68e5e3ca-2621-475d-97e2-b0a891b4abd9") : secret "node-exporter-tls" not found Jul 07 17:28:52.184664 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:52.184561 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:52.294284 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:52.294243 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:52.391863 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:52.391828 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:52.835616 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:52.835575 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:52.900747 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:52.900711 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:53.073345 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:53.073317 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:28:53.111306 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:53.111213 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-tls\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:53.114789 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:53.114759 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68e5e3ca-2621-475d-97e2-b0a891b4abd9-node-exporter-tls\") pod \"node-exporter-dncvg\" (UID: \"68e5e3ca-2621-475d-97e2-b0a891b4abd9\") " pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:53.140650 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:53.140615 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dncvg" Jul 07 17:28:53.148334 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:53.148303 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:53.248639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:53.248599 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:53.807594 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:53.807556 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:53.908166 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:53.908129 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:54.007780 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:54.007746 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:54.108941 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:54.108893 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:54.272506 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:54.272465 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:54.372238 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:54.372129 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:54.638580 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:54.638513 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:54.739599 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:54.739563 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:56.108359 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:56.108314 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:56.195835 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:56.195779 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:56.321709 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:56.321674 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:56.872523 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:56.872489 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:56.933841 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:56.933805 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:57.285855 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:28:57.285813 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68e5e3ca_2621_475d_97e2_b0a891b4abd9.slice/crio-69793b9cb23afb8b24049a98a0cc2f86ed7c10dc0982dd1f340e82b7416e1fe5 WatchSource:0}: Error finding container 69793b9cb23afb8b24049a98a0cc2f86ed7c10dc0982dd1f340e82b7416e1fe5: Status 404 returned error can't find the container with id 69793b9cb23afb8b24049a98a0cc2f86ed7c10dc0982dd1f340e82b7416e1fe5 Jul 07 17:28:57.322047 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:57.322011 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:57.524004 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:57.523938 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:57.594628 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:57.594582 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:58.110927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:58.110831 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:58.214472 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:58.214438 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:58.242334 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:58.242290 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-mpf9p" event={"ID":"8a560068-6fa5-4afa-96d5-06792a244208","Type":"ContainerStarted","Data":"8d565138b357b0345ff8d0a52921ce3128d3c22776e520cf29cede8892746262"} Jul 07 17:28:58.243943 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:58.243440 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-mpf9p" Jul 07 17:28:58.246617 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:58.246444 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dncvg" event={"ID":"68e5e3ca-2621-475d-97e2-b0a891b4abd9","Type":"ContainerStarted","Data":"69793b9cb23afb8b24049a98a0cc2f86ed7c10dc0982dd1f340e82b7416e1fe5"} Jul 07 17:28:58.257799 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:58.257769 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-mpf9p" Jul 07 17:28:58.266940 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:58.266879 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-mpf9p" podStartSLOduration=2.675709372 podStartE2EDuration="23.266862381s" totalImagesPullingTime="20.591153009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:28:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:28:58.26589728 +0000 UTC m=+94.147533807" watchObservedRunningTime="2026-07-07 17:28:58.266862381 +0000 UTC m=+94.148498885" Jul 07 17:28:59.253603 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:59.253565 2562 generic.go:356] "Generic (PLEG): container finished" podID="68e5e3ca-2621-475d-97e2-b0a891b4abd9" containerID="ef6e80120848a9b3ca4ec0aff86612efdc067342cfc5c1c4b3e2b9dc143e94a5" exitCode=0 Jul 07 17:28:59.254762 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:59.254726 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dncvg" event={"ID":"68e5e3ca-2621-475d-97e2-b0a891b4abd9","Type":"ContainerDied","Data":"ef6e80120848a9b3ca4ec0aff86612efdc067342cfc5c1c4b3e2b9dc143e94a5"} Jul 07 17:28:59.257180 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:59.257152 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7fc965b778-q5jc7"] Jul 07 17:28:59.357331 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:59.357290 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:59.459132 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:59.458466 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:28:59.557284 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:28:59.557248 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:00.261058 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:00.261009 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dncvg" event={"ID":"68e5e3ca-2621-475d-97e2-b0a891b4abd9","Type":"ContainerStarted","Data":"2fea4f482318e6654d9561a585b698e4a1bcbe66aa44b1363ad610431da66684"} Jul 07 17:29:00.261632 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:00.261083 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dncvg" event={"ID":"68e5e3ca-2621-475d-97e2-b0a891b4abd9","Type":"ContainerStarted","Data":"f07196e503e563ac69a2f6490b2e849158e3c25e70bbb878ef91aad7fa144cbe"} Jul 07 17:29:00.285984 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:00.285908 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dncvg" podStartSLOduration=7.661873654 podStartE2EDuration="9.28588257s" totalImagesPullingTime="1.516570788s" totalInitContainerRuntime="107.438128ms" isStatefulPod=true podCreationTimestamp="2026-07-07 17:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:29:00.28393013 +0000 UTC m=+96.165566658" watchObservedRunningTime="2026-07-07 17:29:00.28588257 +0000 UTC m=+96.167519098" Jul 07 17:29:01.375340 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:01.375286 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:01.449212 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:01.449161 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:01.857520 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:01.857485 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:01.949892 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:01.949830 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:02.055786 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:02.055755 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-47448" Jul 07 17:29:04.199694 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:04.199657 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:04.281209 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:04.281165 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:06.354410 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:06.354364 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:06.456390 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:06.456341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:07.122600 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:07.122562 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:07.242661 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:07.242623 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:07.297008 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:07.296932 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g" event={"ID":"419724d5-cabf-4577-8d32-c1476cc8ea03","Type":"ContainerStarted","Data":"2866aa270e3a0751016bd02151e0c18ed7d5674d73fb1eadc1489d35570b99e8"} Jul 07 17:29:07.323825 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:07.323752 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bdb5f7bb4-2zs4g" podStartSLOduration=2.916649647 podStartE2EDuration="32.323725469s" totalImagesPullingTime="29.407075822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:28:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:29:07.321343553 +0000 UTC m=+103.202980092" watchObservedRunningTime="2026-07-07 17:29:07.323725469 +0000 UTC m=+103.205361997" Jul 07 17:29:07.333877 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:07.333830 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:07.439941 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:07.439848 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:07.546534 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:07.546497 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:08.812977 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:08.812944 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:08.915543 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:08.915506 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:10.251061 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:10.251032 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:10.354300 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:10.354264 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:10.930359 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:10.930330 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:10.996639 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:10.996602 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:13.515886 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:13.515853 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:13.618478 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:13.618438 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:13.721698 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:13.721663 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:15.338353 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:15.338316 2562 generic.go:356] "Generic (PLEG): container finished" podID="b398fe77-3362-4eaa-b811-95a3d049fcb5" containerID="65bb35b82a264e6879d5883364aef117ba9a46ec2e9e780f7ae32d25a0b8cba7" exitCode=0 Jul 07 17:29:15.338802 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:15.338382 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9" event={"ID":"b398fe77-3362-4eaa-b811-95a3d049fcb5","Type":"ContainerDied","Data":"65bb35b82a264e6879d5883364aef117ba9a46ec2e9e780f7ae32d25a0b8cba7"} Jul 07 17:29:15.338802 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:15.338711 2562 scope.go:117] "RemoveContainer" containerID="65bb35b82a264e6879d5883364aef117ba9a46ec2e9e780f7ae32d25a0b8cba7" Jul 07 17:29:15.418522 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:15.418478 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:16.345107 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:16.345058 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mr9z9" event={"ID":"b398fe77-3362-4eaa-b811-95a3d049fcb5","Type":"ContainerStarted","Data":"195b02ee5b8f07726f8cdedd94c02644c96db723d60daea296b3e9cd09f2bcef"} Jul 07 17:29:23.813582 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:23.813546 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:23.914722 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:23.914684 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:24.286197 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.286102 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" podUID="e4a2ea0f-ff58-4688-b947-46d4352afc62" containerName="registry" containerID="cri-o://c40825e01839ca48cbfe4a731f0083d3fb09b22be4d91385df901ae60c9ec8a0" gracePeriod=30 Jul 07 17:29:24.580749 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.580716 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:29:24.694210 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.694175 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4a2ea0f-ff58-4688-b947-46d4352afc62-installation-pull-secrets\") pod \"e4a2ea0f-ff58-4688-b947-46d4352afc62\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " Jul 07 17:29:24.694409 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.694235 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgkn8\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-kube-api-access-qgkn8\") pod \"e4a2ea0f-ff58-4688-b947-46d4352afc62\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " Jul 07 17:29:24.694409 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.694255 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4a2ea0f-ff58-4688-b947-46d4352afc62-ca-trust-extracted\") pod \"e4a2ea0f-ff58-4688-b947-46d4352afc62\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " Jul 07 17:29:24.694409 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.694272 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4a2ea0f-ff58-4688-b947-46d4352afc62-image-registry-private-configuration\") pod \"e4a2ea0f-ff58-4688-b947-46d4352afc62\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " Jul 07 17:29:24.694409 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.694303 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-bound-sa-token\") pod \"e4a2ea0f-ff58-4688-b947-46d4352afc62\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " Jul 07 17:29:24.694409 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.694324 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls\") pod \"e4a2ea0f-ff58-4688-b947-46d4352afc62\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " Jul 07 17:29:24.694409 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.694360 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-certificates\") pod \"e4a2ea0f-ff58-4688-b947-46d4352afc62\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " Jul 07 17:29:24.694409 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.694393 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4a2ea0f-ff58-4688-b947-46d4352afc62-trusted-ca\") pod \"e4a2ea0f-ff58-4688-b947-46d4352afc62\" (UID: \"e4a2ea0f-ff58-4688-b947-46d4352afc62\") " Jul 07 17:29:24.694897 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.694863 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e4a2ea0f-ff58-4688-b947-46d4352afc62" (UID: "e4a2ea0f-ff58-4688-b947-46d4352afc62"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:29:24.695061 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.694923 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4a2ea0f-ff58-4688-b947-46d4352afc62-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e4a2ea0f-ff58-4688-b947-46d4352afc62" (UID: "e4a2ea0f-ff58-4688-b947-46d4352afc62"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:29:24.696892 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.696864 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4a2ea0f-ff58-4688-b947-46d4352afc62-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e4a2ea0f-ff58-4688-b947-46d4352afc62" (UID: "e4a2ea0f-ff58-4688-b947-46d4352afc62"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:29:24.697017 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.696960 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e4a2ea0f-ff58-4688-b947-46d4352afc62" (UID: "e4a2ea0f-ff58-4688-b947-46d4352afc62"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:29:24.697410 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.697381 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4a2ea0f-ff58-4688-b947-46d4352afc62-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e4a2ea0f-ff58-4688-b947-46d4352afc62" (UID: "e4a2ea0f-ff58-4688-b947-46d4352afc62"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:29:24.697501 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.697468 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-kube-api-access-qgkn8" (OuterVolumeSpecName: "kube-api-access-qgkn8") pod "e4a2ea0f-ff58-4688-b947-46d4352afc62" (UID: "e4a2ea0f-ff58-4688-b947-46d4352afc62"). InnerVolumeSpecName "kube-api-access-qgkn8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:29:24.697573 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.697538 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e4a2ea0f-ff58-4688-b947-46d4352afc62" (UID: "e4a2ea0f-ff58-4688-b947-46d4352afc62"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:29:24.702886 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.702862 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4a2ea0f-ff58-4688-b947-46d4352afc62-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e4a2ea0f-ff58-4688-b947-46d4352afc62" (UID: "e4a2ea0f-ff58-4688-b947-46d4352afc62"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:29:24.796088 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.796001 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-certificates\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:29:24.796088 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.796034 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4a2ea0f-ff58-4688-b947-46d4352afc62-trusted-ca\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:29:24.796088 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.796044 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4a2ea0f-ff58-4688-b947-46d4352afc62-installation-pull-secrets\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:29:24.796088 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.796053 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgkn8\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-kube-api-access-qgkn8\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:29:24.796088 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.796063 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4a2ea0f-ff58-4688-b947-46d4352afc62-ca-trust-extracted\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:29:24.796088 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.796073 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4a2ea0f-ff58-4688-b947-46d4352afc62-image-registry-private-configuration\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:29:24.796088 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.796083 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-bound-sa-token\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:29:24.796088 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:24.796091 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a2ea0f-ff58-4688-b947-46d4352afc62-registry-tls\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:29:25.379452 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:25.379417 2562 generic.go:356] "Generic (PLEG): container finished" podID="e4a2ea0f-ff58-4688-b947-46d4352afc62" containerID="c40825e01839ca48cbfe4a731f0083d3fb09b22be4d91385df901ae60c9ec8a0" exitCode=0 Jul 07 17:29:25.379899 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:25.379468 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" Jul 07 17:29:25.379899 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:25.379504 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" event={"ID":"e4a2ea0f-ff58-4688-b947-46d4352afc62","Type":"ContainerDied","Data":"c40825e01839ca48cbfe4a731f0083d3fb09b22be4d91385df901ae60c9ec8a0"} Jul 07 17:29:25.379899 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:25.379551 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fc965b778-q5jc7" event={"ID":"e4a2ea0f-ff58-4688-b947-46d4352afc62","Type":"ContainerDied","Data":"02254ab35d999962f04d27dbcbcf0d90a392496be7640d125ff3eaf52681ee24"} Jul 07 17:29:25.379899 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:25.379570 2562 scope.go:117] "RemoveContainer" containerID="c40825e01839ca48cbfe4a731f0083d3fb09b22be4d91385df901ae60c9ec8a0" Jul 07 17:29:25.381198 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:25.381177 2562 generic.go:356] "Generic (PLEG): container finished" podID="15b2d389-16ed-4641-bffb-91298e7516c2" containerID="0cafaeb6107307fd71df4d4e856079f7935b50479261e78e0248444c563a588c" exitCode=0 Jul 07 17:29:25.381316 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:25.381219 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" event={"ID":"15b2d389-16ed-4641-bffb-91298e7516c2","Type":"ContainerDied","Data":"0cafaeb6107307fd71df4d4e856079f7935b50479261e78e0248444c563a588c"} Jul 07 17:29:25.381540 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:25.381524 2562 scope.go:117] "RemoveContainer" containerID="0cafaeb6107307fd71df4d4e856079f7935b50479261e78e0248444c563a588c" Jul 07 17:29:25.388534 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:25.388516 2562 scope.go:117] "RemoveContainer" containerID="c40825e01839ca48cbfe4a731f0083d3fb09b22be4d91385df901ae60c9ec8a0" Jul 07 17:29:25.388847 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:29:25.388825 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c40825e01839ca48cbfe4a731f0083d3fb09b22be4d91385df901ae60c9ec8a0\": container with ID starting with c40825e01839ca48cbfe4a731f0083d3fb09b22be4d91385df901ae60c9ec8a0 not found: ID does not exist" containerID="c40825e01839ca48cbfe4a731f0083d3fb09b22be4d91385df901ae60c9ec8a0" Jul 07 17:29:25.388911 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:25.388856 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c40825e01839ca48cbfe4a731f0083d3fb09b22be4d91385df901ae60c9ec8a0"} err="failed to get container status \"c40825e01839ca48cbfe4a731f0083d3fb09b22be4d91385df901ae60c9ec8a0\": rpc error: code = NotFound desc = could not find container \"c40825e01839ca48cbfe4a731f0083d3fb09b22be4d91385df901ae60c9ec8a0\": container with ID starting with c40825e01839ca48cbfe4a731f0083d3fb09b22be4d91385df901ae60c9ec8a0 not found: ID does not exist" Jul 07 17:29:25.425355 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:25.425322 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7fc965b778-q5jc7"] Jul 07 17:29:25.428626 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:25.428599 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7fc965b778-q5jc7"] Jul 07 17:29:26.387703 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:26.387652 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-5gff8" event={"ID":"15b2d389-16ed-4641-bffb-91298e7516c2","Type":"ContainerStarted","Data":"836f0da715f22acabc8b80cb108d19e67d4e9755efb9c7a57f3245dc11edaf62"} Jul 07 17:29:26.637272 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:26.637238 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4a2ea0f-ff58-4688-b947-46d4352afc62" path="/var/lib/kubelet/pods/e4a2ea0f-ff58-4688-b947-46d4352afc62/volumes" Jul 07 17:29:35.425780 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:35.425743 2562 generic.go:356] "Generic (PLEG): container finished" podID="c72df9fa-884f-4fac-8557-199256e8c242" containerID="124a957e9b0a1c82a488be9a025fb5a5d8ffc1310b9efe3479097da8c289c9e7" exitCode=0 Jul 07 17:29:35.426237 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:35.425816 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv" event={"ID":"c72df9fa-884f-4fac-8557-199256e8c242","Type":"ContainerDied","Data":"124a957e9b0a1c82a488be9a025fb5a5d8ffc1310b9efe3479097da8c289c9e7"} Jul 07 17:29:35.426237 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:35.426157 2562 scope.go:117] "RemoveContainer" containerID="124a957e9b0a1c82a488be9a025fb5a5d8ffc1310b9efe3479097da8c289c9e7" Jul 07 17:29:36.432321 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:36.432291 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hrfnv" event={"ID":"c72df9fa-884f-4fac-8557-199256e8c242","Type":"ContainerStarted","Data":"ea08ddbc29d0598bee703646aabe09d02de4488ebf5a8425015d69cbdc34c074"} Jul 07 17:29:50.325631 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:50.325588 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:29:50.392572 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:29:50.392532 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:30:07.466417 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:30:07.466380 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:30:07.570680 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:30:07.570638 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:30:13.211861 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:30:13.211822 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:30:13.275955 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:30:13.275911 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:30:19.390579 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:30:19.390485 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:30:19.458225 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:30:19.458185 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:30:23.751517 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:30:23.751483 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:30:23.890771 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:30:23.890736 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:30:23.958960 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:30:23.958912 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:30:56.369960 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:30:56.369919 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:31:13.680472 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:31:13.680429 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:31:13.780181 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:31:13.780149 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:31:40.429407 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:31:40.429365 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:31:41.426330 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:31:41.426297 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:31:44.227636 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:31:44.227604 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:31:44.328300 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:31:44.328265 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:31:46.232361 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:31:46.232259 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:31:46.427104 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:31:46.427064 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:31:46.612941 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:31:46.612903 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:31:48.584220 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:31:48.584186 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:31:48.899647 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:31:48.899558 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:31:51.440371 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:31:51.440335 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:31:51.540510 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:31:51.540474 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:31:55.938261 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:31:55.938220 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:31:56.023529 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:31:56.023486 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:31:56.297462 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:31:56.297367 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:02.187771 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:02.187731 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:04.490380 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:04.490348 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:07.245495 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:07.245450 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:07.315369 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:07.315320 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:14.202116 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:14.202074 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:14.300775 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:14.300739 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:19.825762 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:19.825719 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:19.924199 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:19.924166 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:22.008474 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:22.008443 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:24.506760 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:24.506731 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 17:32:24.508440 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:24.508416 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 17:32:24.524652 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:24.524616 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 17:32:30.370339 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:30.370300 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:31.914131 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:31.914086 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:36.461715 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:36.461675 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:36.567141 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:36.567103 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:37.067367 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.067328 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:37.300305 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.300272 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:37.398916 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.398820 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:37.462109 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.462077 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8"] Jul 07 17:32:37.462510 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.462375 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4a2ea0f-ff58-4688-b947-46d4352afc62" containerName="registry" Jul 07 17:32:37.462510 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.462391 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4a2ea0f-ff58-4688-b947-46d4352afc62" containerName="registry" Jul 07 17:32:37.462510 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.462461 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4a2ea0f-ff58-4688-b947-46d4352afc62" containerName="registry" Jul 07 17:32:37.464375 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.464357 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.468183 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.468153 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 17:32:37.468313 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.468268 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 17:32:37.468614 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.468596 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 17:32:37.468859 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.468844 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 17:32:37.468922 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.468878 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hbjds\"" Jul 07 17:32:37.484193 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.484167 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8"] Jul 07 17:32:37.527727 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.527689 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/38bb6dde-3263-4924-b9e0-935aefe95a56-manifests\") pod \"opendatahub-operator-controller-manager-88cd6769f-5jms8\" (UID: \"38bb6dde-3263-4924-b9e0-935aefe95a56\") " pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.527727 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.527728 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38bb6dde-3263-4924-b9e0-935aefe95a56-tmp\") pod \"opendatahub-operator-controller-manager-88cd6769f-5jms8\" (UID: \"38bb6dde-3263-4924-b9e0-935aefe95a56\") " pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.527925 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.527767 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6f54\" (UniqueName: \"kubernetes.io/projected/38bb6dde-3263-4924-b9e0-935aefe95a56-kube-api-access-g6f54\") pod \"opendatahub-operator-controller-manager-88cd6769f-5jms8\" (UID: \"38bb6dde-3263-4924-b9e0-935aefe95a56\") " pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.527925 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.527847 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38bb6dde-3263-4924-b9e0-935aefe95a56-webhook-cert\") pod \"opendatahub-operator-controller-manager-88cd6769f-5jms8\" (UID: \"38bb6dde-3263-4924-b9e0-935aefe95a56\") " pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.527925 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.527872 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/38bb6dde-3263-4924-b9e0-935aefe95a56-apiservice-cert\") pod \"opendatahub-operator-controller-manager-88cd6769f-5jms8\" (UID: \"38bb6dde-3263-4924-b9e0-935aefe95a56\") " pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.628779 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.628743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38bb6dde-3263-4924-b9e0-935aefe95a56-webhook-cert\") pod \"opendatahub-operator-controller-manager-88cd6769f-5jms8\" (UID: \"38bb6dde-3263-4924-b9e0-935aefe95a56\") " pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.629052 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.628786 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/38bb6dde-3263-4924-b9e0-935aefe95a56-apiservice-cert\") pod \"opendatahub-operator-controller-manager-88cd6769f-5jms8\" (UID: \"38bb6dde-3263-4924-b9e0-935aefe95a56\") " pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.629052 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.628837 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/38bb6dde-3263-4924-b9e0-935aefe95a56-manifests\") pod \"opendatahub-operator-controller-manager-88cd6769f-5jms8\" (UID: \"38bb6dde-3263-4924-b9e0-935aefe95a56\") " pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.629052 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.628864 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38bb6dde-3263-4924-b9e0-935aefe95a56-tmp\") pod \"opendatahub-operator-controller-manager-88cd6769f-5jms8\" (UID: \"38bb6dde-3263-4924-b9e0-935aefe95a56\") " pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.629052 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.628907 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6f54\" (UniqueName: \"kubernetes.io/projected/38bb6dde-3263-4924-b9e0-935aefe95a56-kube-api-access-g6f54\") pod \"opendatahub-operator-controller-manager-88cd6769f-5jms8\" (UID: \"38bb6dde-3263-4924-b9e0-935aefe95a56\") " pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.629321 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.629293 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/38bb6dde-3263-4924-b9e0-935aefe95a56-manifests\") pod \"opendatahub-operator-controller-manager-88cd6769f-5jms8\" (UID: \"38bb6dde-3263-4924-b9e0-935aefe95a56\") " pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.631412 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.631376 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38bb6dde-3263-4924-b9e0-935aefe95a56-tmp\") pod \"opendatahub-operator-controller-manager-88cd6769f-5jms8\" (UID: \"38bb6dde-3263-4924-b9e0-935aefe95a56\") " pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.631627 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.631608 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/38bb6dde-3263-4924-b9e0-935aefe95a56-apiservice-cert\") pod \"opendatahub-operator-controller-manager-88cd6769f-5jms8\" (UID: \"38bb6dde-3263-4924-b9e0-935aefe95a56\") " pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.631709 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.631673 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38bb6dde-3263-4924-b9e0-935aefe95a56-webhook-cert\") pod \"opendatahub-operator-controller-manager-88cd6769f-5jms8\" (UID: \"38bb6dde-3263-4924-b9e0-935aefe95a56\") " pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.640909 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.640882 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6f54\" (UniqueName: \"kubernetes.io/projected/38bb6dde-3263-4924-b9e0-935aefe95a56-kube-api-access-g6f54\") pod \"opendatahub-operator-controller-manager-88cd6769f-5jms8\" (UID: \"38bb6dde-3263-4924-b9e0-935aefe95a56\") " pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.666251 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.666157 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:37.760034 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.760001 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:37.774828 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.774796 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:37.865348 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.865305 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:37.918592 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.918563 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8"] Jul 07 17:32:37.921562 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:32:37.921536 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38bb6dde_3263_4924_b9e0_935aefe95a56.slice/crio-90a77745d8ff1e9506a52eb5b33a6b1b0440b78acae1f702394fd89aaf5fcf3f WatchSource:0}: Error finding container 90a77745d8ff1e9506a52eb5b33a6b1b0440b78acae1f702394fd89aaf5fcf3f: Status 404 returned error can't find the container with id 90a77745d8ff1e9506a52eb5b33a6b1b0440b78acae1f702394fd89aaf5fcf3f Jul 07 17:32:37.923454 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.923436 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 17:32:37.941054 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:37.941018 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:38.011276 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:38.011233 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:38.118223 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:38.118188 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:38.149442 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:38.149407 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" event={"ID":"38bb6dde-3263-4924-b9e0-935aefe95a56","Type":"ContainerStarted","Data":"90a77745d8ff1e9506a52eb5b33a6b1b0440b78acae1f702394fd89aaf5fcf3f"} Jul 07 17:32:41.165952 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:41.165866 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" event={"ID":"38bb6dde-3263-4924-b9e0-935aefe95a56","Type":"ContainerStarted","Data":"32876cda675df3cc39628821eacb1f5d38d140d9d31dd622314c74c3eb079c0b"} Jul 07 17:32:42.171803 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:42.171767 2562 generic.go:356] "Generic (PLEG): container finished" podID="38bb6dde-3263-4924-b9e0-935aefe95a56" containerID="32876cda675df3cc39628821eacb1f5d38d140d9d31dd622314c74c3eb079c0b" exitCode=0 Jul 07 17:32:42.172202 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:42.171840 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" event={"ID":"38bb6dde-3263-4924-b9e0-935aefe95a56","Type":"ContainerDied","Data":"32876cda675df3cc39628821eacb1f5d38d140d9d31dd622314c74c3eb079c0b"} Jul 07 17:32:43.177879 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:43.177842 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" event={"ID":"38bb6dde-3263-4924-b9e0-935aefe95a56","Type":"ContainerStarted","Data":"ab9e519efac689da7fccfbd0c50ecf1343ebd3e875947063489ffb93fad35244"} Jul 07 17:32:43.178265 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:43.177935 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:43.214582 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:43.214529 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" podStartSLOduration=1.727722058 podStartE2EDuration="6.214511324s" totalImagesPullingTime="3.482109037s" totalInitContainerRuntime="1.004680229s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:32:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:32:43.212704871 +0000 UTC m=+319.094341394" watchObservedRunningTime="2026-07-07 17:32:43.214511324 +0000 UTC m=+319.096147850" Jul 07 17:32:54.184005 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:54.183956 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-88cd6769f-5jms8" Jul 07 17:32:56.188748 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:56.188707 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:56.292145 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:56.292109 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:57.052444 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.052407 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:57.265535 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.265499 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:57.447559 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.447530 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg"] Jul 07 17:32:57.455834 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.455811 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg" Jul 07 17:32:57.461368 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.461338 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 17:32:57.461544 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.461375 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 17:32:57.462897 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.462870 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pfhlb\"" Jul 07 17:32:57.471054 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.471031 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg"] Jul 07 17:32:57.499699 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.499661 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f3e8cac-099e-4860-8b63-6d27ab9477aa-tls-certs\") pod \"kube-auth-proxy-bbcb8bd4c-hkrzg\" (UID: \"6f3e8cac-099e-4860-8b63-6d27ab9477aa\") " pod="openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg" Jul 07 17:32:57.499881 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.499737 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f3e8cac-099e-4860-8b63-6d27ab9477aa-tmp\") pod \"kube-auth-proxy-bbcb8bd4c-hkrzg\" (UID: \"6f3e8cac-099e-4860-8b63-6d27ab9477aa\") " pod="openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg" Jul 07 17:32:57.499881 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.499774 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5j2k\" (UniqueName: \"kubernetes.io/projected/6f3e8cac-099e-4860-8b63-6d27ab9477aa-kube-api-access-k5j2k\") pod \"kube-auth-proxy-bbcb8bd4c-hkrzg\" (UID: \"6f3e8cac-099e-4860-8b63-6d27ab9477aa\") " pod="openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg" Jul 07 17:32:57.551983 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.551930 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8kp6k"] Jul 07 17:32:57.554346 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.554327 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-8kp6k" Jul 07 17:32:57.562472 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.562446 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-ft9fr\"" Jul 07 17:32:57.562589 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.562491 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 07 17:32:57.571491 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.571453 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8kp6k"] Jul 07 17:32:57.600950 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.600908 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f3e8cac-099e-4860-8b63-6d27ab9477aa-tls-certs\") pod \"kube-auth-proxy-bbcb8bd4c-hkrzg\" (UID: \"6f3e8cac-099e-4860-8b63-6d27ab9477aa\") " pod="openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg" Jul 07 17:32:57.600950 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.600948 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh7bg\" (UniqueName: \"kubernetes.io/projected/df5a9647-84dc-4717-91e0-0a8cb94da704-kube-api-access-qh7bg\") pod \"odh-model-controller-7998c67dc4-8kp6k\" (UID: \"df5a9647-84dc-4717-91e0-0a8cb94da704\") " pod="opendatahub/odh-model-controller-7998c67dc4-8kp6k" Jul 07 17:32:57.601194 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.601004 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/df5a9647-84dc-4717-91e0-0a8cb94da704-cert\") pod \"odh-model-controller-7998c67dc4-8kp6k\" (UID: \"df5a9647-84dc-4717-91e0-0a8cb94da704\") " pod="opendatahub/odh-model-controller-7998c67dc4-8kp6k" Jul 07 17:32:57.601194 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.601059 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f3e8cac-099e-4860-8b63-6d27ab9477aa-tmp\") pod \"kube-auth-proxy-bbcb8bd4c-hkrzg\" (UID: \"6f3e8cac-099e-4860-8b63-6d27ab9477aa\") " pod="openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg" Jul 07 17:32:57.601194 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.601152 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5j2k\" (UniqueName: \"kubernetes.io/projected/6f3e8cac-099e-4860-8b63-6d27ab9477aa-kube-api-access-k5j2k\") pod \"kube-auth-proxy-bbcb8bd4c-hkrzg\" (UID: \"6f3e8cac-099e-4860-8b63-6d27ab9477aa\") " pod="openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg" Jul 07 17:32:57.603528 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.603500 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f3e8cac-099e-4860-8b63-6d27ab9477aa-tmp\") pod \"kube-auth-proxy-bbcb8bd4c-hkrzg\" (UID: \"6f3e8cac-099e-4860-8b63-6d27ab9477aa\") " pod="openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg" Jul 07 17:32:57.603725 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.603693 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f3e8cac-099e-4860-8b63-6d27ab9477aa-tls-certs\") pod \"kube-auth-proxy-bbcb8bd4c-hkrzg\" (UID: \"6f3e8cac-099e-4860-8b63-6d27ab9477aa\") " pod="openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg" Jul 07 17:32:57.611766 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.611733 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:57.624622 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.624595 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5j2k\" (UniqueName: \"kubernetes.io/projected/6f3e8cac-099e-4860-8b63-6d27ab9477aa-kube-api-access-k5j2k\") pod \"kube-auth-proxy-bbcb8bd4c-hkrzg\" (UID: \"6f3e8cac-099e-4860-8b63-6d27ab9477aa\") " pod="openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg" Jul 07 17:32:57.702623 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.702520 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh7bg\" (UniqueName: \"kubernetes.io/projected/df5a9647-84dc-4717-91e0-0a8cb94da704-kube-api-access-qh7bg\") pod \"odh-model-controller-7998c67dc4-8kp6k\" (UID: \"df5a9647-84dc-4717-91e0-0a8cb94da704\") " pod="opendatahub/odh-model-controller-7998c67dc4-8kp6k" Jul 07 17:32:57.702623 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.702576 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/df5a9647-84dc-4717-91e0-0a8cb94da704-cert\") pod \"odh-model-controller-7998c67dc4-8kp6k\" (UID: \"df5a9647-84dc-4717-91e0-0a8cb94da704\") " pod="opendatahub/odh-model-controller-7998c67dc4-8kp6k" Jul 07 17:32:57.702855 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:32:57.702730 2562 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 07 17:32:57.702855 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:32:57.702790 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df5a9647-84dc-4717-91e0-0a8cb94da704-cert podName:df5a9647-84dc-4717-91e0-0a8cb94da704 nodeName:}" failed. No retries permitted until 2026-07-07 17:32:58.202771278 +0000 UTC m=+334.084407783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/df5a9647-84dc-4717-91e0-0a8cb94da704-cert") pod "odh-model-controller-7998c67dc4-8kp6k" (UID: "df5a9647-84dc-4717-91e0-0a8cb94da704") : secret "odh-model-controller-webhook-cert" not found Jul 07 17:32:57.715843 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.715816 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh7bg\" (UniqueName: \"kubernetes.io/projected/df5a9647-84dc-4717-91e0-0a8cb94da704-kube-api-access-qh7bg\") pod \"odh-model-controller-7998c67dc4-8kp6k\" (UID: \"df5a9647-84dc-4717-91e0-0a8cb94da704\") " pod="opendatahub/odh-model-controller-7998c67dc4-8kp6k" Jul 07 17:32:57.732617 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.732581 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:57.765078 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.765049 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg" Jul 07 17:32:57.838164 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.838106 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:57.922804 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.922772 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg"] Jul 07 17:32:57.926497 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:32:57.926471 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f3e8cac_099e_4860_8b63_6d27ab9477aa.slice/crio-1d02f49079ff54c932308e6c713f1ee3767284b754ece7ed52360871fc0afdf4 WatchSource:0}: Error finding container 1d02f49079ff54c932308e6c713f1ee3767284b754ece7ed52360871fc0afdf4: Status 404 returned error can't find the container with id 1d02f49079ff54c932308e6c713f1ee3767284b754ece7ed52360871fc0afdf4 Jul 07 17:32:57.936044 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:57.936017 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:58.206983 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:58.206941 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/df5a9647-84dc-4717-91e0-0a8cb94da704-cert\") pod \"odh-model-controller-7998c67dc4-8kp6k\" (UID: \"df5a9647-84dc-4717-91e0-0a8cb94da704\") " pod="opendatahub/odh-model-controller-7998c67dc4-8kp6k" Jul 07 17:32:58.209583 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:58.209563 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/df5a9647-84dc-4717-91e0-0a8cb94da704-cert\") pod \"odh-model-controller-7998c67dc4-8kp6k\" (UID: \"df5a9647-84dc-4717-91e0-0a8cb94da704\") " pod="opendatahub/odh-model-controller-7998c67dc4-8kp6k" Jul 07 17:32:58.245624 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:58.245592 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg" event={"ID":"6f3e8cac-099e-4860-8b63-6d27ab9477aa","Type":"ContainerStarted","Data":"1d02f49079ff54c932308e6c713f1ee3767284b754ece7ed52360871fc0afdf4"} Jul 07 17:32:58.465050 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:58.464933 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-8kp6k" Jul 07 17:32:58.629423 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:58.629394 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8kp6k"] Jul 07 17:32:58.629916 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:32:58.629885 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf5a9647_84dc_4717_91e0_0a8cb94da704.slice/crio-832b6bc79c6faeb3bc4f6362e622a843210f72b5e7a4d3c660b5229b5e219f6a WatchSource:0}: Error finding container 832b6bc79c6faeb3bc4f6362e622a843210f72b5e7a4d3c660b5229b5e219f6a: Status 404 returned error can't find the container with id 832b6bc79c6faeb3bc4f6362e622a843210f72b5e7a4d3c660b5229b5e219f6a Jul 07 17:32:58.806623 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:58.806539 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:58.896225 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:58.896171 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-z2fhk"] Jul 07 17:32:58.898979 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:58.898937 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-z2fhk" Jul 07 17:32:58.909379 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:58.909352 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-xbkvg\"" Jul 07 17:32:58.910711 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:58.910690 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 07 17:32:58.926804 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:58.926436 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-z2fhk"] Jul 07 17:32:58.991470 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:58.991437 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:59.014359 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.014321 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72269\" (UniqueName: \"kubernetes.io/projected/7de9ab54-6d76-466f-99d9-d4dadaf41c15-kube-api-access-72269\") pod \"model-serving-api-54d89bd5fc-z2fhk\" (UID: \"7de9ab54-6d76-466f-99d9-d4dadaf41c15\") " pod="opendatahub/model-serving-api-54d89bd5fc-z2fhk" Jul 07 17:32:59.014359 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.014365 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7de9ab54-6d76-466f-99d9-d4dadaf41c15-tls-certs\") pod \"model-serving-api-54d89bd5fc-z2fhk\" (UID: \"7de9ab54-6d76-466f-99d9-d4dadaf41c15\") " pod="opendatahub/model-serving-api-54d89bd5fc-z2fhk" Jul 07 17:32:59.115201 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.115159 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72269\" (UniqueName: \"kubernetes.io/projected/7de9ab54-6d76-466f-99d9-d4dadaf41c15-kube-api-access-72269\") pod \"model-serving-api-54d89bd5fc-z2fhk\" (UID: \"7de9ab54-6d76-466f-99d9-d4dadaf41c15\") " pod="opendatahub/model-serving-api-54d89bd5fc-z2fhk" Jul 07 17:32:59.115404 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.115223 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7de9ab54-6d76-466f-99d9-d4dadaf41c15-tls-certs\") pod \"model-serving-api-54d89bd5fc-z2fhk\" (UID: \"7de9ab54-6d76-466f-99d9-d4dadaf41c15\") " pod="opendatahub/model-serving-api-54d89bd5fc-z2fhk" Jul 07 17:32:59.118048 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.118024 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7de9ab54-6d76-466f-99d9-d4dadaf41c15-tls-certs\") pod \"model-serving-api-54d89bd5fc-z2fhk\" (UID: \"7de9ab54-6d76-466f-99d9-d4dadaf41c15\") " pod="opendatahub/model-serving-api-54d89bd5fc-z2fhk" Jul 07 17:32:59.129929 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.129899 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72269\" (UniqueName: \"kubernetes.io/projected/7de9ab54-6d76-466f-99d9-d4dadaf41c15-kube-api-access-72269\") pod \"model-serving-api-54d89bd5fc-z2fhk\" (UID: \"7de9ab54-6d76-466f-99d9-d4dadaf41c15\") " pod="opendatahub/model-serving-api-54d89bd5fc-z2fhk" Jul 07 17:32:59.212989 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.212925 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-z2fhk" Jul 07 17:32:59.254692 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.254654 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-8kp6k" event={"ID":"df5a9647-84dc-4717-91e0-0a8cb94da704","Type":"ContainerStarted","Data":"832b6bc79c6faeb3bc4f6362e622a843210f72b5e7a4d3c660b5229b5e219f6a"} Jul 07 17:32:59.398988 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.398693 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2"] Jul 07 17:32:59.402320 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.402292 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-z2fhk"] Jul 07 17:32:59.402447 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.402420 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.408101 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.408079 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 17:32:59.408277 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.408253 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 17:32:59.408360 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.408288 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 17:32:59.408520 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.408503 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-tbx7c\"" Jul 07 17:32:59.408644 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.408081 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:59.408733 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.408717 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 17:32:59.409409 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:32:59.409371 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7de9ab54_6d76_466f_99d9_d4dadaf41c15.slice/crio-13f9fde0271d915471cd951ff55af900440e92f7a33a2384352b94022bd6832d WatchSource:0}: Error finding container 13f9fde0271d915471cd951ff55af900440e92f7a33a2384352b94022bd6832d: Status 404 returned error can't find the container with id 13f9fde0271d915471cd951ff55af900440e92f7a33a2384352b94022bd6832d Jul 07 17:32:59.424891 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.424847 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2"] Jul 07 17:32:59.519075 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.519025 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.519569 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.519106 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/933e5b6c-0181-4f01-86ee-008943b759c0-local-certs\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.519569 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.519133 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.519569 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.519197 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/933e5b6c-0181-4f01-86ee-008943b759c0-istio-token\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.519569 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.519222 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-cacerts\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.519569 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.519250 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/933e5b6c-0181-4f01-86ee-008943b759c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.519569 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.519274 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-946sl\" (UniqueName: \"kubernetes.io/projected/933e5b6c-0181-4f01-86ee-008943b759c0-kube-api-access-946sl\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.532212 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.532180 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:32:59.619915 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.619877 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/933e5b6c-0181-4f01-86ee-008943b759c0-local-certs\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.620215 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.619935 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.620215 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.620016 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/933e5b6c-0181-4f01-86ee-008943b759c0-istio-token\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.620215 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.620041 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-cacerts\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.620215 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.620070 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/933e5b6c-0181-4f01-86ee-008943b759c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.620215 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.620094 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-946sl\" (UniqueName: \"kubernetes.io/projected/933e5b6c-0181-4f01-86ee-008943b759c0-kube-api-access-946sl\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.620215 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.620177 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.621406 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.621374 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/933e5b6c-0181-4f01-86ee-008943b759c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.623956 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.623927 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.624518 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.624476 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/933e5b6c-0181-4f01-86ee-008943b759c0-local-certs\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.626657 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.626630 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-cacerts\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.627037 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.626997 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.635582 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.635525 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/933e5b6c-0181-4f01-86ee-008943b759c0-istio-token\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.647365 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.647327 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-946sl\" (UniqueName: \"kubernetes.io/projected/933e5b6c-0181-4f01-86ee-008943b759c0-kube-api-access-946sl\") pod \"istiod-openshift-gateway-7574459656-xj9w2\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.718265 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.718175 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:32:59.911349 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:32:59.911258 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2"] Jul 07 17:33:00.261130 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:00.261085 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-z2fhk" event={"ID":"7de9ab54-6d76-466f-99d9-d4dadaf41c15","Type":"ContainerStarted","Data":"13f9fde0271d915471cd951ff55af900440e92f7a33a2384352b94022bd6832d"} Jul 07 17:33:00.693073 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:33:00.693024 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod933e5b6c_0181_4f01_86ee_008943b759c0.slice/crio-39f0c945cbaf08cc72f035292fe8f5ba964d660d669c1b39bfe08d15cac11622 WatchSource:0}: Error finding container 39f0c945cbaf08cc72f035292fe8f5ba964d660d669c1b39bfe08d15cac11622: Status 404 returned error can't find the container with id 39f0c945cbaf08cc72f035292fe8f5ba964d660d669c1b39bfe08d15cac11622 Jul 07 17:33:00.946930 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:00.946717 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:01.268328 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:01.268230 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" event={"ID":"933e5b6c-0181-4f01-86ee-008943b759c0","Type":"ContainerStarted","Data":"39f0c945cbaf08cc72f035292fe8f5ba964d660d669c1b39bfe08d15cac11622"} Jul 07 17:33:03.280889 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:03.280850 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-8kp6k" event={"ID":"df5a9647-84dc-4717-91e0-0a8cb94da704","Type":"ContainerStarted","Data":"0f0488161f055865a7506334a4c5f1069827f9694b6a952a4282fc232194324a"} Jul 07 17:33:03.281397 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:03.280921 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-8kp6k" Jul 07 17:33:03.282688 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:03.282630 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-z2fhk" event={"ID":"7de9ab54-6d76-466f-99d9-d4dadaf41c15","Type":"ContainerStarted","Data":"48319f59e2626b5d769a6e2a79dcb153566d8e5cdea1d0f80a1d734597a1a432"} Jul 07 17:33:03.282821 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:03.282748 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-z2fhk" Jul 07 17:33:03.284437 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:03.284413 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg" event={"ID":"6f3e8cac-099e-4860-8b63-6d27ab9477aa","Type":"ContainerStarted","Data":"ff5694a7ca4213d57058d50101b997a3e6758ba4212414139e5280df5f9bde13"} Jul 07 17:33:03.307626 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:03.307571 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-8kp6k" podStartSLOduration=2.631109108 podStartE2EDuration="6.307551501s" totalImagesPullingTime="3.676442393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:32:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:33:03.305518404 +0000 UTC m=+339.187154932" watchObservedRunningTime="2026-07-07 17:33:03.307551501 +0000 UTC m=+339.189188029" Jul 07 17:33:03.334772 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:03.334712 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-bbcb8bd4c-hkrzg" podStartSLOduration=1.9649637850000001 podStartE2EDuration="6.334687456s" totalImagesPullingTime="4.369723671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:32:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:33:03.333043283 +0000 UTC m=+339.214679809" watchObservedRunningTime="2026-07-07 17:33:03.334687456 +0000 UTC m=+339.216323984" Jul 07 17:33:03.433756 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:03.433700 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-z2fhk" podStartSLOduration=2.546485088 podStartE2EDuration="5.43367646s" totalImagesPullingTime="2.887191372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:33:03.431993872 +0000 UTC m=+339.313630377" watchObservedRunningTime="2026-07-07 17:33:03.43367646 +0000 UTC m=+339.315312986" Jul 07 17:33:04.278303 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:04.278258 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 07 17:33:04.278391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:04.278340 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 07 17:33:05.294885 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:05.294842 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" event={"ID":"933e5b6c-0181-4f01-86ee-008943b759c0","Type":"ContainerStarted","Data":"85cf849a89ad3830b7c626376583f64e2b015db763cb2dba4273f0636c7a07e6"} Jul 07 17:33:05.295395 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:05.295043 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:33:05.297211 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:05.297187 2562 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-xj9w2 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 17:33:05.297353 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:05.297236 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" podUID="933e5b6c-0181-4f01-86ee-008943b759c0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 17:33:05.494084 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:05.494034 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:05.579621 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:05.579589 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:06.301518 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:06.301485 2562 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-xj9w2 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 17:33:06.301876 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:06.301542 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" podUID="933e5b6c-0181-4f01-86ee-008943b759c0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 17:33:06.705730 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:06.705692 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:06.965008 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:06.964894 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:07.081378 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:07.081341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:09.066304 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.066269 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:09.199280 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.199235 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:09.248926 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.248870 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" podStartSLOduration=6.666806048 podStartE2EDuration="10.248850559s" totalImagesPullingTime="3.582044511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:32:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:33:05.329104458 +0000 UTC m=+341.210740987" watchObservedRunningTime="2026-07-07 17:33:09.248850559 +0000 UTC m=+345.130487086" Jul 07 17:33:09.249946 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.249929 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jxgfh"] Jul 07 17:33:09.252363 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.252347 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxgfh" Jul 07 17:33:09.258563 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.258540 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 07 17:33:09.258563 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.258542 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-tpmbx\"" Jul 07 17:33:09.302516 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.302473 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:33:09.304077 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.304057 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-h7kbp"] Jul 07 17:33:09.306142 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.306126 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h7kbp" Jul 07 17:33:09.310020 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.309993 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jxgfh"] Jul 07 17:33:09.314598 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.314578 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-cls56\"" Jul 07 17:33:09.314743 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.314719 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 07 17:33:09.321430 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.321366 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-h7kbp"] Jul 07 17:33:09.331768 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.331710 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:09.408655 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.408612 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4294403d-0f91-4e28-974c-e7e725239890-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-h7kbp\" (UID: \"4294403d-0f91-4e28-974c-e7e725239890\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h7kbp" Jul 07 17:33:09.408852 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.408779 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c97f57d4-0f45-4ddb-b9eb-8ee38f89dca1-cert\") pod \"kserve-controller-manager-d6f4b65b8-jxgfh\" (UID: \"c97f57d4-0f45-4ddb-b9eb-8ee38f89dca1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxgfh" Jul 07 17:33:09.408852 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.408836 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5bs9\" (UniqueName: \"kubernetes.io/projected/c97f57d4-0f45-4ddb-b9eb-8ee38f89dca1-kube-api-access-g5bs9\") pod \"kserve-controller-manager-d6f4b65b8-jxgfh\" (UID: \"c97f57d4-0f45-4ddb-b9eb-8ee38f89dca1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxgfh" Jul 07 17:33:09.409008 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.408944 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q8vq\" (UniqueName: \"kubernetes.io/projected/4294403d-0f91-4e28-974c-e7e725239890-kube-api-access-6q8vq\") pod \"llmisvc-controller-manager-777c6bd7d5-h7kbp\" (UID: \"4294403d-0f91-4e28-974c-e7e725239890\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h7kbp" Jul 07 17:33:09.431862 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.431830 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2"] Jul 07 17:33:09.432055 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.432014 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" podUID="933e5b6c-0181-4f01-86ee-008943b759c0" containerName="discovery" containerID="cri-o://85cf849a89ad3830b7c626376583f64e2b015db763cb2dba4273f0636c7a07e6" gracePeriod=30 Jul 07 17:33:09.488092 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.488063 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:09.510136 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.510097 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q8vq\" (UniqueName: \"kubernetes.io/projected/4294403d-0f91-4e28-974c-e7e725239890-kube-api-access-6q8vq\") pod \"llmisvc-controller-manager-777c6bd7d5-h7kbp\" (UID: \"4294403d-0f91-4e28-974c-e7e725239890\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h7kbp" Jul 07 17:33:09.510136 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.510147 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4294403d-0f91-4e28-974c-e7e725239890-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-h7kbp\" (UID: \"4294403d-0f91-4e28-974c-e7e725239890\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h7kbp" Jul 07 17:33:09.510372 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.510190 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c97f57d4-0f45-4ddb-b9eb-8ee38f89dca1-cert\") pod \"kserve-controller-manager-d6f4b65b8-jxgfh\" (UID: \"c97f57d4-0f45-4ddb-b9eb-8ee38f89dca1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxgfh" Jul 07 17:33:09.510372 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.510216 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5bs9\" (UniqueName: \"kubernetes.io/projected/c97f57d4-0f45-4ddb-b9eb-8ee38f89dca1-kube-api-access-g5bs9\") pod \"kserve-controller-manager-d6f4b65b8-jxgfh\" (UID: \"c97f57d4-0f45-4ddb-b9eb-8ee38f89dca1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxgfh" Jul 07 17:33:09.512792 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.512768 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c97f57d4-0f45-4ddb-b9eb-8ee38f89dca1-cert\") pod \"kserve-controller-manager-d6f4b65b8-jxgfh\" (UID: \"c97f57d4-0f45-4ddb-b9eb-8ee38f89dca1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxgfh" Jul 07 17:33:09.512896 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.512814 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4294403d-0f91-4e28-974c-e7e725239890-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-h7kbp\" (UID: \"4294403d-0f91-4e28-974c-e7e725239890\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h7kbp" Jul 07 17:33:09.530593 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.530559 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5bs9\" (UniqueName: \"kubernetes.io/projected/c97f57d4-0f45-4ddb-b9eb-8ee38f89dca1-kube-api-access-g5bs9\") pod \"kserve-controller-manager-d6f4b65b8-jxgfh\" (UID: \"c97f57d4-0f45-4ddb-b9eb-8ee38f89dca1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxgfh" Jul 07 17:33:09.532621 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.532603 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q8vq\" (UniqueName: \"kubernetes.io/projected/4294403d-0f91-4e28-974c-e7e725239890-kube-api-access-6q8vq\") pod \"llmisvc-controller-manager-777c6bd7d5-h7kbp\" (UID: \"4294403d-0f91-4e28-974c-e7e725239890\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h7kbp" Jul 07 17:33:09.563948 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.563908 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxgfh" Jul 07 17:33:09.580610 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.580579 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:09.620767 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.619671 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h7kbp" Jul 07 17:33:09.677089 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.677055 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:09.682061 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.681400 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:33:09.738652 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.738623 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jxgfh"] Jul 07 17:33:09.741396 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:33:09.741363 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc97f57d4_0f45_4ddb_b9eb_8ee38f89dca1.slice/crio-1da16c0dc3ab0218fdc5cdc5fbb31a814918d081d68ebf2530a0bed7a0ca9534 WatchSource:0}: Error finding container 1da16c0dc3ab0218fdc5cdc5fbb31a814918d081d68ebf2530a0bed7a0ca9534: Status 404 returned error can't find the container with id 1da16c0dc3ab0218fdc5cdc5fbb31a814918d081d68ebf2530a0bed7a0ca9534 Jul 07 17:33:09.804138 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.804113 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-h7kbp"] Jul 07 17:33:09.806201 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:33:09.806171 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4294403d_0f91_4e28_974c_e7e725239890.slice/crio-7254d030bdf0964881987686faf656aa706a74b2519fc8da9804deb7655de356 WatchSource:0}: Error finding container 7254d030bdf0964881987686faf656aa706a74b2519fc8da9804deb7655de356: Status 404 returned error can't find the container with id 7254d030bdf0964881987686faf656aa706a74b2519fc8da9804deb7655de356 Jul 07 17:33:09.813609 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.813581 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/933e5b6c-0181-4f01-86ee-008943b759c0-local-certs\") pod \"933e5b6c-0181-4f01-86ee-008943b759c0\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " Jul 07 17:33:09.813690 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.813633 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-istio-csr-dns-cert\") pod \"933e5b6c-0181-4f01-86ee-008943b759c0\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " Jul 07 17:33:09.813690 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.813678 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/933e5b6c-0181-4f01-86ee-008943b759c0-istio-token\") pod \"933e5b6c-0181-4f01-86ee-008943b759c0\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " Jul 07 17:33:09.813763 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.813724 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/933e5b6c-0181-4f01-86ee-008943b759c0-istio-csr-ca-configmap\") pod \"933e5b6c-0181-4f01-86ee-008943b759c0\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " Jul 07 17:33:09.813804 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.813762 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-946sl\" (UniqueName: \"kubernetes.io/projected/933e5b6c-0181-4f01-86ee-008943b759c0-kube-api-access-946sl\") pod \"933e5b6c-0181-4f01-86ee-008943b759c0\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " Jul 07 17:33:09.813804 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.813792 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-istio-kubeconfig\") pod \"933e5b6c-0181-4f01-86ee-008943b759c0\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " Jul 07 17:33:09.813891 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.813852 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-cacerts\") pod \"933e5b6c-0181-4f01-86ee-008943b759c0\" (UID: \"933e5b6c-0181-4f01-86ee-008943b759c0\") " Jul 07 17:33:09.814178 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.814126 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/933e5b6c-0181-4f01-86ee-008943b759c0-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "933e5b6c-0181-4f01-86ee-008943b759c0" (UID: "933e5b6c-0181-4f01-86ee-008943b759c0"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:33:09.816079 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.816049 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "933e5b6c-0181-4f01-86ee-008943b759c0" (UID: "933e5b6c-0181-4f01-86ee-008943b759c0"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:33:09.816193 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.816054 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933e5b6c-0181-4f01-86ee-008943b759c0-local-certs" (OuterVolumeSpecName: "local-certs") pod "933e5b6c-0181-4f01-86ee-008943b759c0" (UID: "933e5b6c-0181-4f01-86ee-008943b759c0"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:33:09.816360 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.816337 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/933e5b6c-0181-4f01-86ee-008943b759c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "933e5b6c-0181-4f01-86ee-008943b759c0" (UID: "933e5b6c-0181-4f01-86ee-008943b759c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:33:09.816515 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.816499 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-cacerts" (OuterVolumeSpecName: "cacerts") pod "933e5b6c-0181-4f01-86ee-008943b759c0" (UID: "933e5b6c-0181-4f01-86ee-008943b759c0"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:33:09.816614 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.816593 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/933e5b6c-0181-4f01-86ee-008943b759c0-kube-api-access-946sl" (OuterVolumeSpecName: "kube-api-access-946sl") pod "933e5b6c-0181-4f01-86ee-008943b759c0" (UID: "933e5b6c-0181-4f01-86ee-008943b759c0"). InnerVolumeSpecName "kube-api-access-946sl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:33:09.816614 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.816606 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "933e5b6c-0181-4f01-86ee-008943b759c0" (UID: "933e5b6c-0181-4f01-86ee-008943b759c0"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:33:09.914505 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.914421 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-946sl\" (UniqueName: \"kubernetes.io/projected/933e5b6c-0181-4f01-86ee-008943b759c0-kube-api-access-946sl\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:33:09.914505 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.914447 2562 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-istio-kubeconfig\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:33:09.914505 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.914456 2562 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-cacerts\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:33:09.914505 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.914464 2562 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/933e5b6c-0181-4f01-86ee-008943b759c0-local-certs\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:33:09.914505 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.914472 2562 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/933e5b6c-0181-4f01-86ee-008943b759c0-istio-csr-dns-cert\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:33:09.914505 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.914480 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/933e5b6c-0181-4f01-86ee-008943b759c0-istio-token\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:33:09.914505 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:09.914490 2562 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/933e5b6c-0181-4f01-86ee-008943b759c0-istio-csr-ca-configmap\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:33:10.324515 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:10.324480 2562 generic.go:356] "Generic (PLEG): container finished" podID="933e5b6c-0181-4f01-86ee-008943b759c0" containerID="85cf849a89ad3830b7c626376583f64e2b015db763cb2dba4273f0636c7a07e6" exitCode=0 Jul 07 17:33:10.324924 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:10.324551 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" Jul 07 17:33:10.324924 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:10.324566 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" event={"ID":"933e5b6c-0181-4f01-86ee-008943b759c0","Type":"ContainerDied","Data":"85cf849a89ad3830b7c626376583f64e2b015db763cb2dba4273f0636c7a07e6"} Jul 07 17:33:10.324924 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:10.324639 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2" event={"ID":"933e5b6c-0181-4f01-86ee-008943b759c0","Type":"ContainerDied","Data":"39f0c945cbaf08cc72f035292fe8f5ba964d660d669c1b39bfe08d15cac11622"} Jul 07 17:33:10.324924 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:10.324671 2562 scope.go:117] "RemoveContainer" containerID="85cf849a89ad3830b7c626376583f64e2b015db763cb2dba4273f0636c7a07e6" Jul 07 17:33:10.325920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:10.325892 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h7kbp" event={"ID":"4294403d-0f91-4e28-974c-e7e725239890","Type":"ContainerStarted","Data":"7254d030bdf0964881987686faf656aa706a74b2519fc8da9804deb7655de356"} Jul 07 17:33:10.327266 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:10.327241 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxgfh" event={"ID":"c97f57d4-0f45-4ddb-b9eb-8ee38f89dca1","Type":"ContainerStarted","Data":"1da16c0dc3ab0218fdc5cdc5fbb31a814918d081d68ebf2530a0bed7a0ca9534"} Jul 07 17:33:10.334319 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:10.334300 2562 scope.go:117] "RemoveContainer" containerID="85cf849a89ad3830b7c626376583f64e2b015db763cb2dba4273f0636c7a07e6" Jul 07 17:33:10.334675 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:33:10.334643 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85cf849a89ad3830b7c626376583f64e2b015db763cb2dba4273f0636c7a07e6\": container with ID starting with 85cf849a89ad3830b7c626376583f64e2b015db763cb2dba4273f0636c7a07e6 not found: ID does not exist" containerID="85cf849a89ad3830b7c626376583f64e2b015db763cb2dba4273f0636c7a07e6" Jul 07 17:33:10.334773 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:10.334683 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85cf849a89ad3830b7c626376583f64e2b015db763cb2dba4273f0636c7a07e6"} err="failed to get container status \"85cf849a89ad3830b7c626376583f64e2b015db763cb2dba4273f0636c7a07e6\": rpc error: code = NotFound desc = could not find container \"85cf849a89ad3830b7c626376583f64e2b015db763cb2dba4273f0636c7a07e6\": container with ID starting with 85cf849a89ad3830b7c626376583f64e2b015db763cb2dba4273f0636c7a07e6 not found: ID does not exist" Jul 07 17:33:10.361392 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:10.361359 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2"] Jul 07 17:33:10.367314 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:10.367278 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xj9w2"] Jul 07 17:33:10.639283 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:10.639210 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="933e5b6c-0181-4f01-86ee-008943b759c0" path="/var/lib/kubelet/pods/933e5b6c-0181-4f01-86ee-008943b759c0/volumes" Jul 07 17:33:13.344617 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:13.344577 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h7kbp" event={"ID":"4294403d-0f91-4e28-974c-e7e725239890","Type":"ContainerStarted","Data":"5fb4883bec363a3f92189d72cae1964b89d86c6c08b6026b44b74ab6b1639127"} Jul 07 17:33:13.345114 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:13.344676 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h7kbp" Jul 07 17:33:13.346279 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:13.346253 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxgfh" event={"ID":"c97f57d4-0f45-4ddb-b9eb-8ee38f89dca1","Type":"ContainerStarted","Data":"f4a0c7a311b75ae46e5812e74635bcc79d9d3b35c9e787f182712732b23705e1"} Jul 07 17:33:13.346394 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:13.346299 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxgfh" Jul 07 17:33:13.389582 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:13.389520 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h7kbp" podStartSLOduration=1.100408985 podStartE2EDuration="4.389504423s" totalImagesPullingTime="3.289095438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:33:13.388314984 +0000 UTC m=+349.269951524" watchObservedRunningTime="2026-07-07 17:33:13.389504423 +0000 UTC m=+349.271140949" Jul 07 17:33:14.291082 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:14.291047 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-8kp6k" Jul 07 17:33:14.292942 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:14.292920 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-z2fhk" Jul 07 17:33:14.321666 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:14.321600 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxgfh" podStartSLOduration=2.7044486340000002 podStartE2EDuration="5.321583077s" totalImagesPullingTime="2.617134443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:33:13.431823708 +0000 UTC m=+349.313460235" watchObservedRunningTime="2026-07-07 17:33:14.321583077 +0000 UTC m=+350.203219604" Jul 07 17:33:44.353651 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:44.353621 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h7kbp" Jul 07 17:33:44.356997 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:44.356958 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxgfh" Jul 07 17:33:50.977725 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:50.977679 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:51.038091 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:51.038057 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:57.613738 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:57.613702 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:57.710207 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:57.710174 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:57.752374 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:57.752312 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gw92g"] Jul 07 17:33:57.752696 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:57.752682 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="933e5b6c-0181-4f01-86ee-008943b759c0" containerName="discovery" Jul 07 17:33:57.752739 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:57.752698 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="933e5b6c-0181-4f01-86ee-008943b759c0" containerName="discovery" Jul 07 17:33:57.752806 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:57.752796 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="933e5b6c-0181-4f01-86ee-008943b759c0" containerName="discovery" Jul 07 17:33:57.755181 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:57.755154 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gw92g" Jul 07 17:33:57.758058 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:57.758030 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 17:33:57.759208 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:57.759168 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-ntnbm\"" Jul 07 17:33:57.759208 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:57.759200 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 17:33:57.767264 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:57.767235 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gw92g"] Jul 07 17:33:57.804135 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:57.804099 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5667n\" (UniqueName: \"kubernetes.io/projected/69ce380e-1a6d-471c-af41-60791370ef0d-kube-api-access-5667n\") pod \"kuadrant-operator-catalog-gw92g\" (UID: \"69ce380e-1a6d-471c-af41-60791370ef0d\") " pod="kuadrant-system/kuadrant-operator-catalog-gw92g" Jul 07 17:33:57.825503 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:57.825469 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:33:57.905381 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:57.905290 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5667n\" (UniqueName: \"kubernetes.io/projected/69ce380e-1a6d-471c-af41-60791370ef0d-kube-api-access-5667n\") pod \"kuadrant-operator-catalog-gw92g\" (UID: \"69ce380e-1a6d-471c-af41-60791370ef0d\") " pod="kuadrant-system/kuadrant-operator-catalog-gw92g" Jul 07 17:33:57.915739 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:57.915712 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5667n\" (UniqueName: \"kubernetes.io/projected/69ce380e-1a6d-471c-af41-60791370ef0d-kube-api-access-5667n\") pod \"kuadrant-operator-catalog-gw92g\" (UID: \"69ce380e-1a6d-471c-af41-60791370ef0d\") " pod="kuadrant-system/kuadrant-operator-catalog-gw92g" Jul 07 17:33:58.066112 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:58.066059 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gw92g" Jul 07 17:33:58.089367 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:58.089340 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gw92g"] Jul 07 17:33:58.228981 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:58.228945 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gw92g"] Jul 07 17:33:58.230945 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:33:58.230913 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ce380e_1a6d_471c_af41_60791370ef0d.slice/crio-81d1090315cbaa7ff29782b79b71e2d84630217f274cdf614e8b6d657d79ab6d WatchSource:0}: Error finding container 81d1090315cbaa7ff29782b79b71e2d84630217f274cdf614e8b6d657d79ab6d: Status 404 returned error can't find the container with id 81d1090315cbaa7ff29782b79b71e2d84630217f274cdf614e8b6d657d79ab6d Jul 07 17:33:58.300115 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:58.300081 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-t96jf"] Jul 07 17:33:58.302925 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:58.302908 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-t96jf" Jul 07 17:33:58.313505 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:58.313480 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-t96jf"] Jul 07 17:33:58.409293 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:58.409261 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndlg8\" (UniqueName: \"kubernetes.io/projected/e8d55d8b-1c52-421d-9120-332ce2f8f2f5-kube-api-access-ndlg8\") pod \"kuadrant-operator-catalog-t96jf\" (UID: \"e8d55d8b-1c52-421d-9120-332ce2f8f2f5\") " pod="kuadrant-system/kuadrant-operator-catalog-t96jf" Jul 07 17:33:58.510703 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:58.510599 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndlg8\" (UniqueName: \"kubernetes.io/projected/e8d55d8b-1c52-421d-9120-332ce2f8f2f5-kube-api-access-ndlg8\") pod \"kuadrant-operator-catalog-t96jf\" (UID: \"e8d55d8b-1c52-421d-9120-332ce2f8f2f5\") " pod="kuadrant-system/kuadrant-operator-catalog-t96jf" Jul 07 17:33:58.521384 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:58.521353 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndlg8\" (UniqueName: \"kubernetes.io/projected/e8d55d8b-1c52-421d-9120-332ce2f8f2f5-kube-api-access-ndlg8\") pod \"kuadrant-operator-catalog-t96jf\" (UID: \"e8d55d8b-1c52-421d-9120-332ce2f8f2f5\") " pod="kuadrant-system/kuadrant-operator-catalog-t96jf" Jul 07 17:33:58.563203 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:58.563166 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gw92g" event={"ID":"69ce380e-1a6d-471c-af41-60791370ef0d","Type":"ContainerStarted","Data":"81d1090315cbaa7ff29782b79b71e2d84630217f274cdf614e8b6d657d79ab6d"} Jul 07 17:33:58.612944 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:58.612903 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-t96jf" Jul 07 17:33:58.758051 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:58.758028 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-t96jf"] Jul 07 17:33:58.759219 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:33:58.759187 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8d55d8b_1c52_421d_9120_332ce2f8f2f5.slice/crio-78a622f7a59dbb8a24f37f416bc94162f7ebdd0a5fff72abeac033a55a00113d WatchSource:0}: Error finding container 78a622f7a59dbb8a24f37f416bc94162f7ebdd0a5fff72abeac033a55a00113d: Status 404 returned error can't find the container with id 78a622f7a59dbb8a24f37f416bc94162f7ebdd0a5fff72abeac033a55a00113d Jul 07 17:33:59.569552 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:33:59.569515 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-t96jf" event={"ID":"e8d55d8b-1c52-421d-9120-332ce2f8f2f5","Type":"ContainerStarted","Data":"78a622f7a59dbb8a24f37f416bc94162f7ebdd0a5fff72abeac033a55a00113d"} Jul 07 17:34:00.575232 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:00.575203 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gw92g" event={"ID":"69ce380e-1a6d-471c-af41-60791370ef0d","Type":"ContainerStarted","Data":"f90ba40011c8ccfc283edd7c96ea12c7dddcd25230f761e07d457146d9fdc383"} Jul 07 17:34:00.575710 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:00.575260 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-gw92g" podUID="69ce380e-1a6d-471c-af41-60791370ef0d" containerName="registry-server" containerID="cri-o://f90ba40011c8ccfc283edd7c96ea12c7dddcd25230f761e07d457146d9fdc383" gracePeriod=2 Jul 07 17:34:00.576920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:00.576896 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-t96jf" event={"ID":"e8d55d8b-1c52-421d-9120-332ce2f8f2f5","Type":"ContainerStarted","Data":"f693446e8dd506fe4f3247314e3b3dd3c03d3c68ae04f51d683678e78f459d25"} Jul 07 17:34:00.595800 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:00.595755 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-gw92g" podStartSLOduration=1.602435346 podStartE2EDuration="3.595737964s" totalImagesPullingTime="1.993302618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:33:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:34:00.594315348 +0000 UTC m=+396.475951877" watchObservedRunningTime="2026-07-07 17:34:00.595737964 +0000 UTC m=+396.477374492" Jul 07 17:34:00.613488 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:00.613437 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-t96jf" podStartSLOduration=1.145639119 podStartE2EDuration="2.613421504s" totalImagesPullingTime="1.467782385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:33:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:34:00.612912115 +0000 UTC m=+396.494548647" watchObservedRunningTime="2026-07-07 17:34:00.613421504 +0000 UTC m=+396.495058030" Jul 07 17:34:00.820655 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:00.820620 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gw92g" Jul 07 17:34:00.932546 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:00.932463 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5667n\" (UniqueName: \"kubernetes.io/projected/69ce380e-1a6d-471c-af41-60791370ef0d-kube-api-access-5667n\") pod \"69ce380e-1a6d-471c-af41-60791370ef0d\" (UID: \"69ce380e-1a6d-471c-af41-60791370ef0d\") " Jul 07 17:34:00.934822 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:00.934788 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69ce380e-1a6d-471c-af41-60791370ef0d-kube-api-access-5667n" (OuterVolumeSpecName: "kube-api-access-5667n") pod "69ce380e-1a6d-471c-af41-60791370ef0d" (UID: "69ce380e-1a6d-471c-af41-60791370ef0d"). InnerVolumeSpecName "kube-api-access-5667n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:34:01.034113 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:01.034075 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5667n\" (UniqueName: \"kubernetes.io/projected/69ce380e-1a6d-471c-af41-60791370ef0d-kube-api-access-5667n\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:34:01.583072 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:01.583035 2562 generic.go:356] "Generic (PLEG): container finished" podID="69ce380e-1a6d-471c-af41-60791370ef0d" containerID="f90ba40011c8ccfc283edd7c96ea12c7dddcd25230f761e07d457146d9fdc383" exitCode=0 Jul 07 17:34:01.583478 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:01.583079 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gw92g" Jul 07 17:34:01.583478 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:01.583121 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gw92g" event={"ID":"69ce380e-1a6d-471c-af41-60791370ef0d","Type":"ContainerDied","Data":"f90ba40011c8ccfc283edd7c96ea12c7dddcd25230f761e07d457146d9fdc383"} Jul 07 17:34:01.583478 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:01.583170 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gw92g" event={"ID":"69ce380e-1a6d-471c-af41-60791370ef0d","Type":"ContainerDied","Data":"81d1090315cbaa7ff29782b79b71e2d84630217f274cdf614e8b6d657d79ab6d"} Jul 07 17:34:01.583478 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:01.583195 2562 scope.go:117] "RemoveContainer" containerID="f90ba40011c8ccfc283edd7c96ea12c7dddcd25230f761e07d457146d9fdc383" Jul 07 17:34:01.592732 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:01.592710 2562 scope.go:117] "RemoveContainer" containerID="f90ba40011c8ccfc283edd7c96ea12c7dddcd25230f761e07d457146d9fdc383" Jul 07 17:34:01.593117 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:34:01.593094 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f90ba40011c8ccfc283edd7c96ea12c7dddcd25230f761e07d457146d9fdc383\": container with ID starting with f90ba40011c8ccfc283edd7c96ea12c7dddcd25230f761e07d457146d9fdc383 not found: ID does not exist" containerID="f90ba40011c8ccfc283edd7c96ea12c7dddcd25230f761e07d457146d9fdc383" Jul 07 17:34:01.593204 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:01.593128 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f90ba40011c8ccfc283edd7c96ea12c7dddcd25230f761e07d457146d9fdc383"} err="failed to get container status \"f90ba40011c8ccfc283edd7c96ea12c7dddcd25230f761e07d457146d9fdc383\": rpc error: code = NotFound desc = could not find container \"f90ba40011c8ccfc283edd7c96ea12c7dddcd25230f761e07d457146d9fdc383\": container with ID starting with f90ba40011c8ccfc283edd7c96ea12c7dddcd25230f761e07d457146d9fdc383 not found: ID does not exist" Jul 07 17:34:01.610764 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:01.610729 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gw92g"] Jul 07 17:34:01.616239 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:01.616207 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gw92g"] Jul 07 17:34:02.637814 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:02.637776 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69ce380e-1a6d-471c-af41-60791370ef0d" path="/var/lib/kubelet/pods/69ce380e-1a6d-471c-af41-60791370ef0d/volumes" Jul 07 17:34:08.613410 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:08.613371 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-t96jf" Jul 07 17:34:08.613854 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:08.613439 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-t96jf" Jul 07 17:34:08.638251 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:08.638224 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-t96jf" Jul 07 17:34:09.642502 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:09.642472 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-t96jf" Jul 07 17:34:21.587213 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:21.587174 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:24.244062 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:24.244020 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz"] Jul 07 17:34:24.244469 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:24.244433 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69ce380e-1a6d-471c-af41-60791370ef0d" containerName="registry-server" Jul 07 17:34:24.244469 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:24.244446 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="69ce380e-1a6d-471c-af41-60791370ef0d" containerName="registry-server" Jul 07 17:34:24.244546 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:24.244519 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="69ce380e-1a6d-471c-af41-60791370ef0d" containerName="registry-server" Jul 07 17:34:24.246641 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:24.246622 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" Jul 07 17:34:24.249732 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:24.249709 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-tb25t\"" Jul 07 17:34:24.259883 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:24.259859 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz"] Jul 07 17:34:24.431780 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:24.431744 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlzvf\" (UniqueName: \"kubernetes.io/projected/ebb2b9e6-d1ea-454f-8daf-91b6052edc26-kube-api-access-nlzvf\") pod \"limitador-operator-controller-manager-859d7b78bd-wxlkz\" (UID: \"ebb2b9e6-d1ea-454f-8daf-91b6052edc26\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" Jul 07 17:34:24.532522 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:24.532428 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlzvf\" (UniqueName: \"kubernetes.io/projected/ebb2b9e6-d1ea-454f-8daf-91b6052edc26-kube-api-access-nlzvf\") pod \"limitador-operator-controller-manager-859d7b78bd-wxlkz\" (UID: \"ebb2b9e6-d1ea-454f-8daf-91b6052edc26\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" Jul 07 17:34:24.554546 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:24.554521 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlzvf\" (UniqueName: \"kubernetes.io/projected/ebb2b9e6-d1ea-454f-8daf-91b6052edc26-kube-api-access-nlzvf\") pod \"limitador-operator-controller-manager-859d7b78bd-wxlkz\" (UID: \"ebb2b9e6-d1ea-454f-8daf-91b6052edc26\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" Jul 07 17:34:24.560112 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:24.560090 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-tb25t\"" Jul 07 17:34:24.567657 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:24.567637 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" Jul 07 17:34:24.725869 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:34:24.725836 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebb2b9e6_d1ea_454f_8daf_91b6052edc26.slice/crio-62d71715dd28951ad355db5e1ba47fbd35a17b8c472ede9fc09013130748c13b WatchSource:0}: Error finding container 62d71715dd28951ad355db5e1ba47fbd35a17b8c472ede9fc09013130748c13b: Status 404 returned error can't find the container with id 62d71715dd28951ad355db5e1ba47fbd35a17b8c472ede9fc09013130748c13b Jul 07 17:34:24.732352 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:24.732327 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz"] Jul 07 17:34:25.203306 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:25.203269 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:25.306697 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:25.306644 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:25.691722 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:25.691685 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" event={"ID":"ebb2b9e6-d1ea-454f-8daf-91b6052edc26","Type":"ContainerStarted","Data":"62d71715dd28951ad355db5e1ba47fbd35a17b8c472ede9fc09013130748c13b"} Jul 07 17:34:26.697466 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:26.697361 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" event={"ID":"ebb2b9e6-d1ea-454f-8daf-91b6052edc26","Type":"ContainerStarted","Data":"bde4c0417f7eefe63b5b7a3c7d6b0839e71c0c3a271a172704a4b935f499f358"} Jul 07 17:34:26.697466 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:26.697441 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" Jul 07 17:34:26.721184 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:26.721132 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" podStartSLOduration=1.083633103 podStartE2EDuration="2.721117466s" totalImagesPullingTime="1.637484363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:34:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:34:26.719407449 +0000 UTC m=+422.601043978" watchObservedRunningTime="2026-07-07 17:34:26.721117466 +0000 UTC m=+422.602753990" Jul 07 17:34:29.486110 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:29.486073 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn"] Jul 07 17:34:29.492653 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:29.492629 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" Jul 07 17:34:29.495712 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:29.495690 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4tcmv\"" Jul 07 17:34:29.505300 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:29.505277 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn"] Jul 07 17:34:29.579094 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:29.579058 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2v5b\" (UniqueName: \"kubernetes.io/projected/5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84-kube-api-access-g2v5b\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6fbsn\" (UID: \"5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" Jul 07 17:34:29.579260 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:29.579122 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6fbsn\" (UID: \"5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" Jul 07 17:34:29.679591 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:29.679545 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2v5b\" (UniqueName: \"kubernetes.io/projected/5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84-kube-api-access-g2v5b\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6fbsn\" (UID: \"5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" Jul 07 17:34:29.679802 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:29.679604 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6fbsn\" (UID: \"5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" Jul 07 17:34:29.679952 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:29.679931 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6fbsn\" (UID: \"5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" Jul 07 17:34:29.693620 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:29.693586 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2v5b\" (UniqueName: \"kubernetes.io/projected/5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84-kube-api-access-g2v5b\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6fbsn\" (UID: \"5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" Jul 07 17:34:29.787687 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:29.787590 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:29.803846 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:29.803813 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" Jul 07 17:34:29.957633 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:29.957608 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn"] Jul 07 17:34:29.960148 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:34:29.960119 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b1c5a44_c2a1_4fb1_a3d6_b5bec2d55e84.slice/crio-159c3ff2fdcd36b5441f1b6a95d2742a60bedcb7817931c130ac8014ffa18fb4 WatchSource:0}: Error finding container 159c3ff2fdcd36b5441f1b6a95d2742a60bedcb7817931c130ac8014ffa18fb4: Status 404 returned error can't find the container with id 159c3ff2fdcd36b5441f1b6a95d2742a60bedcb7817931c130ac8014ffa18fb4 Jul 07 17:34:30.210671 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:30.210640 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:30.717464 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:30.717429 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" event={"ID":"5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84","Type":"ContainerStarted","Data":"159c3ff2fdcd36b5441f1b6a95d2742a60bedcb7817931c130ac8014ffa18fb4"} Jul 07 17:34:31.642055 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:31.642019 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:32.171269 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:32.171234 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pfh22"] Jul 07 17:34:32.174756 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:32.174734 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pfh22" Jul 07 17:34:32.178545 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:32.178517 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-fwb6k\"" Jul 07 17:34:32.187612 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:32.187582 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pfh22"] Jul 07 17:34:32.198377 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:32.198337 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txstd\" (UniqueName: \"kubernetes.io/projected/cae879d3-65ad-4ef6-8991-2d755cd3e40a-kube-api-access-txstd\") pod \"authorino-operator-75cc8d5cb7-pfh22\" (UID: \"cae879d3-65ad-4ef6-8991-2d755cd3e40a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pfh22" Jul 07 17:34:32.299593 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:32.299547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txstd\" (UniqueName: \"kubernetes.io/projected/cae879d3-65ad-4ef6-8991-2d755cd3e40a-kube-api-access-txstd\") pod \"authorino-operator-75cc8d5cb7-pfh22\" (UID: \"cae879d3-65ad-4ef6-8991-2d755cd3e40a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pfh22" Jul 07 17:34:32.311452 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:32.311424 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txstd\" (UniqueName: \"kubernetes.io/projected/cae879d3-65ad-4ef6-8991-2d755cd3e40a-kube-api-access-txstd\") pod \"authorino-operator-75cc8d5cb7-pfh22\" (UID: \"cae879d3-65ad-4ef6-8991-2d755cd3e40a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pfh22" Jul 07 17:34:32.487405 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:32.487320 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pfh22" Jul 07 17:34:32.645644 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:32.645623 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pfh22"] Jul 07 17:34:32.647677 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:34:32.647650 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae879d3_65ad_4ef6_8991_2d755cd3e40a.slice/crio-3fcf4da0f8a67109123d8fa418b0d1bca9f72e2033fc9be4c70b39b73afd93dd WatchSource:0}: Error finding container 3fcf4da0f8a67109123d8fa418b0d1bca9f72e2033fc9be4c70b39b73afd93dd: Status 404 returned error can't find the container with id 3fcf4da0f8a67109123d8fa418b0d1bca9f72e2033fc9be4c70b39b73afd93dd Jul 07 17:34:32.727998 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:32.727936 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pfh22" event={"ID":"cae879d3-65ad-4ef6-8991-2d755cd3e40a","Type":"ContainerStarted","Data":"3fcf4da0f8a67109123d8fa418b0d1bca9f72e2033fc9be4c70b39b73afd93dd"} Jul 07 17:34:33.228493 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:33.228431 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:34.110542 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:34.110508 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:34.200238 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:34.200202 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:35.748067 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:35.748029 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pfh22" event={"ID":"cae879d3-65ad-4ef6-8991-2d755cd3e40a","Type":"ContainerStarted","Data":"2d7fb10734486fd6576dab2917a4e38d63042a334e1e971272e78f8d3e968b18"} Jul 07 17:34:35.748499 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:35.748155 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pfh22" Jul 07 17:34:35.770301 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:35.770243 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pfh22" podStartSLOduration=1.5187679360000002 podStartE2EDuration="3.77022235s" totalImagesPullingTime="2.251454414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:34:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:34:35.768993648 +0000 UTC m=+431.650630358" watchObservedRunningTime="2026-07-07 17:34:35.77022235 +0000 UTC m=+431.651858877" Jul 07 17:34:36.037818 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:36.037717 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:36.755495 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:36.755456 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" event={"ID":"5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84","Type":"ContainerStarted","Data":"c791c267fa8092c04df574ce48c6c407059ca2206d08b8fd22830ff57a60d2de"} Jul 07 17:34:36.755946 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:36.755639 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" Jul 07 17:34:36.785346 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:36.785289 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" podStartSLOduration=1.5417316749999999 podStartE2EDuration="7.785270937s" totalImagesPullingTime="6.243539262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:34:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:34:36.785223708 +0000 UTC m=+432.666860235" watchObservedRunningTime="2026-07-07 17:34:36.785270937 +0000 UTC m=+432.666907466" Jul 07 17:34:37.437302 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:37.437264 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:37.528960 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:37.528922 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:37.703648 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:37.703558 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" Jul 07 17:34:39.215494 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:39.215460 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:39.682384 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:39.682341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:40.811602 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:40.811562 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:40.911221 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:40.911183 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:46.758167 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:46.758083 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pfh22" Jul 07 17:34:47.763431 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:47.763396 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" Jul 07 17:34:49.441298 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.441259 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:49.548040 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.547998 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:49.670412 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.670380 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn"] Jul 07 17:34:49.670584 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.670561 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" podUID="5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84" containerName="manager" containerID="cri-o://c791c267fa8092c04df574ce48c6c407059ca2206d08b8fd22830ff57a60d2de" gracePeriod=2 Jul 07 17:34:49.681457 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.681371 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn"] Jul 07 17:34:49.693281 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.693202 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz"] Jul 07 17:34:49.693470 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.693439 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" podUID="ebb2b9e6-d1ea-454f-8daf-91b6052edc26" containerName="manager" containerID="cri-o://bde4c0417f7eefe63b5b7a3c7d6b0839e71c0c3a271a172704a4b935f499f358" gracePeriod=2 Jul 07 17:34:49.708736 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.708703 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7"] Jul 07 17:34:49.709394 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.709367 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84" containerName="manager" Jul 07 17:34:49.709394 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.709393 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84" containerName="manager" Jul 07 17:34:49.709603 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.709476 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84" containerName="manager" Jul 07 17:34:49.711583 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.711562 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz"] Jul 07 17:34:49.711681 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.711659 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" Jul 07 17:34:49.711863 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:34:49.711828 2562 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ebb2b9e6-d1ea-454f-8daf-91b6052edc26\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-wxlkz\": pods \"limitador-operator-controller-manager-859d7b78bd-wxlkz\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" Jul 07 17:34:49.714650 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:34:49.714612 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6fbsn\" is forbidden: User \"system:node:ip-10-0-141-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-75.ec2.internal' and this object" podUID="5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" Jul 07 17:34:49.717043 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:34:49.717013 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6fbsn\" is forbidden: User \"system:node:ip-10-0-141-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-75.ec2.internal' and this object" podUID="5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" Jul 07 17:34:49.735954 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.735919 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7"] Jul 07 17:34:49.755225 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:34:49.755182 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wxlkz\" is forbidden: User \"system:node:ip-10-0-141-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-75.ec2.internal' and this object" podUID="ebb2b9e6-d1ea-454f-8daf-91b6052edc26" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" Jul 07 17:34:49.770620 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.770587 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae6941dc-d3f2-4cf9-bb5d-939837b87997-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7bbb7\" (UID: \"ae6941dc-d3f2-4cf9-bb5d-939837b87997\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" Jul 07 17:34:49.770761 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.770649 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqthf\" (UniqueName: \"kubernetes.io/projected/ae6941dc-d3f2-4cf9-bb5d-939837b87997-kube-api-access-hqthf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7bbb7\" (UID: \"ae6941dc-d3f2-4cf9-bb5d-939837b87997\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" Jul 07 17:34:49.820886 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.820856 2562 generic.go:356] "Generic (PLEG): container finished" podID="ebb2b9e6-d1ea-454f-8daf-91b6052edc26" containerID="bde4c0417f7eefe63b5b7a3c7d6b0839e71c0c3a271a172704a4b935f499f358" exitCode=0 Jul 07 17:34:49.822805 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.822784 2562 generic.go:356] "Generic (PLEG): container finished" podID="5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84" containerID="c791c267fa8092c04df574ce48c6c407059ca2206d08b8fd22830ff57a60d2de" exitCode=0 Jul 07 17:34:49.871694 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.871660 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae6941dc-d3f2-4cf9-bb5d-939837b87997-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7bbb7\" (UID: \"ae6941dc-d3f2-4cf9-bb5d-939837b87997\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" Jul 07 17:34:49.871837 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.871728 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqthf\" (UniqueName: \"kubernetes.io/projected/ae6941dc-d3f2-4cf9-bb5d-939837b87997-kube-api-access-hqthf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7bbb7\" (UID: \"ae6941dc-d3f2-4cf9-bb5d-939837b87997\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" Jul 07 17:34:49.872137 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.872114 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae6941dc-d3f2-4cf9-bb5d-939837b87997-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7bbb7\" (UID: \"ae6941dc-d3f2-4cf9-bb5d-939837b87997\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" Jul 07 17:34:49.882857 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.882823 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqthf\" (UniqueName: \"kubernetes.io/projected/ae6941dc-d3f2-4cf9-bb5d-939837b87997-kube-api-access-hqthf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7bbb7\" (UID: \"ae6941dc-d3f2-4cf9-bb5d-939837b87997\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" Jul 07 17:34:49.941387 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.941361 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" Jul 07 17:34:49.944246 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:34:49.944187 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6fbsn\" is forbidden: User \"system:node:ip-10-0-141-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-75.ec2.internal' and this object" podUID="5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" Jul 07 17:34:49.944383 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:49.944369 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" Jul 07 17:34:49.946519 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:34:49.946500 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wxlkz\" is forbidden: User \"system:node:ip-10-0-141-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-75.ec2.internal' and this object" podUID="ebb2b9e6-d1ea-454f-8daf-91b6052edc26" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" Jul 07 17:34:49.948806 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:34:49.948787 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wxlkz\" is forbidden: User \"system:node:ip-10-0-141-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-75.ec2.internal' and this object" podUID="ebb2b9e6-d1ea-454f-8daf-91b6052edc26" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" Jul 07 17:34:49.950940 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:34:49.950921 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6fbsn\" is forbidden: User \"system:node:ip-10-0-141-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-75.ec2.internal' and this object" podUID="5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" Jul 07 17:34:50.074275 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.074233 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84-extensions-socket-volume\") pod \"5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84\" (UID: \"5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84\") " Jul 07 17:34:50.074452 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.074296 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlzvf\" (UniqueName: \"kubernetes.io/projected/ebb2b9e6-d1ea-454f-8daf-91b6052edc26-kube-api-access-nlzvf\") pod \"ebb2b9e6-d1ea-454f-8daf-91b6052edc26\" (UID: \"ebb2b9e6-d1ea-454f-8daf-91b6052edc26\") " Jul 07 17:34:50.074452 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.074351 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2v5b\" (UniqueName: \"kubernetes.io/projected/5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84-kube-api-access-g2v5b\") pod \"5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84\" (UID: \"5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84\") " Jul 07 17:34:50.074744 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.074716 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84" (UID: "5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:34:50.076589 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.076563 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebb2b9e6-d1ea-454f-8daf-91b6052edc26-kube-api-access-nlzvf" (OuterVolumeSpecName: "kube-api-access-nlzvf") pod "ebb2b9e6-d1ea-454f-8daf-91b6052edc26" (UID: "ebb2b9e6-d1ea-454f-8daf-91b6052edc26"). InnerVolumeSpecName "kube-api-access-nlzvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:34:50.076589 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.076578 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84-kube-api-access-g2v5b" (OuterVolumeSpecName: "kube-api-access-g2v5b") pod "5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84" (UID: "5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84"). InnerVolumeSpecName "kube-api-access-g2v5b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:34:50.119572 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.119533 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" Jul 07 17:34:50.175667 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.175630 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2v5b\" (UniqueName: \"kubernetes.io/projected/5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84-kube-api-access-g2v5b\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:34:50.175804 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.175671 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84-extensions-socket-volume\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:34:50.175804 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.175687 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlzvf\" (UniqueName: \"kubernetes.io/projected/ebb2b9e6-d1ea-454f-8daf-91b6052edc26-kube-api-access-nlzvf\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:34:50.316290 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.316259 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7"] Jul 07 17:34:50.322410 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:34:50.322379 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae6941dc_d3f2_4cf9_bb5d_939837b87997.slice/crio-570b37072bafe492ad76d0e74df7b12addb816fc184b7c4b70da26ddb3a2e211 WatchSource:0}: Error finding container 570b37072bafe492ad76d0e74df7b12addb816fc184b7c4b70da26ddb3a2e211: Status 404 returned error can't find the container with id 570b37072bafe492ad76d0e74df7b12addb816fc184b7c4b70da26ddb3a2e211 Jul 07 17:34:50.638535 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.638501 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84" path="/var/lib/kubelet/pods/5b1c5a44-c2a1-4fb1-a3d6-b5bec2d55e84/volumes" Jul 07 17:34:50.638925 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.638880 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebb2b9e6-d1ea-454f-8daf-91b6052edc26" path="/var/lib/kubelet/pods/ebb2b9e6-d1ea-454f-8daf-91b6052edc26/volumes" Jul 07 17:34:50.829421 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.829392 2562 scope.go:117] "RemoveContainer" containerID="c791c267fa8092c04df574ce48c6c407059ca2206d08b8fd22830ff57a60d2de" Jul 07 17:34:50.829591 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.829397 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fbsn" Jul 07 17:34:50.831237 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.831212 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" event={"ID":"ae6941dc-d3f2-4cf9-bb5d-939837b87997","Type":"ContainerStarted","Data":"40ded52f4f54cf0d5d2547cc842018377b7209c6c468edc79f0d02c4e6095fa1"} Jul 07 17:34:50.831346 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.831273 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" Jul 07 17:34:50.831346 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.831296 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" event={"ID":"ae6941dc-d3f2-4cf9-bb5d-939837b87997","Type":"ContainerStarted","Data":"570b37072bafe492ad76d0e74df7b12addb816fc184b7c4b70da26ddb3a2e211"} Jul 07 17:34:50.832788 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.832771 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wxlkz" Jul 07 17:34:50.839387 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.839356 2562 scope.go:117] "RemoveContainer" containerID="bde4c0417f7eefe63b5b7a3c7d6b0839e71c0c3a271a172704a4b935f499f358" Jul 07 17:34:50.855098 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:50.855028 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" podStartSLOduration=1.85501107 podStartE2EDuration="1.85501107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:34:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:34:50.85265498 +0000 UTC m=+446.734291508" watchObservedRunningTime="2026-07-07 17:34:50.85501107 +0000 UTC m=+446.736647596" Jul 07 17:34:51.888774 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:51.888746 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:51.987980 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:51.987940 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:56.148036 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:56.148002 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:56.248431 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:56.248400 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:58.630322 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:58.630283 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:34:58.736530 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:34:58.736488 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:01.842766 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:01.842736 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" Jul 07 17:35:07.498726 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:07.498687 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:07.604687 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:07.604649 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:09.963795 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:09.963756 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:10.032746 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:10.032707 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:17.380824 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:17.380784 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:17.482803 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:17.482769 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:20.532118 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:20.532081 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:20.784925 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:20.784810 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:20.895746 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:20.895708 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:21.870166 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:21.870134 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7"] Jul 07 17:35:21.870600 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:21.870363 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" podUID="ae6941dc-d3f2-4cf9-bb5d-939837b87997" containerName="manager" containerID="cri-o://40ded52f4f54cf0d5d2547cc842018377b7209c6c468edc79f0d02c4e6095fa1" gracePeriod=10 Jul 07 17:35:21.987056 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:21.987027 2562 generic.go:356] "Generic (PLEG): container finished" podID="ae6941dc-d3f2-4cf9-bb5d-939837b87997" containerID="40ded52f4f54cf0d5d2547cc842018377b7209c6c468edc79f0d02c4e6095fa1" exitCode=0 Jul 07 17:35:21.987200 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:21.987101 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" event={"ID":"ae6941dc-d3f2-4cf9-bb5d-939837b87997","Type":"ContainerDied","Data":"40ded52f4f54cf0d5d2547cc842018377b7209c6c468edc79f0d02c4e6095fa1"} Jul 07 17:35:22.136848 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.136818 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" Jul 07 17:35:22.140594 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.140569 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt"] Jul 07 17:35:22.140918 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.140907 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae6941dc-d3f2-4cf9-bb5d-939837b87997" containerName="manager" Jul 07 17:35:22.140961 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.140920 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae6941dc-d3f2-4cf9-bb5d-939837b87997" containerName="manager" Jul 07 17:35:22.140961 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.140939 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebb2b9e6-d1ea-454f-8daf-91b6052edc26" containerName="manager" Jul 07 17:35:22.140961 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.140945 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebb2b9e6-d1ea-454f-8daf-91b6052edc26" containerName="manager" Jul 07 17:35:22.141066 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.141031 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae6941dc-d3f2-4cf9-bb5d-939837b87997" containerName="manager" Jul 07 17:35:22.141066 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.141041 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebb2b9e6-d1ea-454f-8daf-91b6052edc26" containerName="manager" Jul 07 17:35:22.142815 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.142801 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" Jul 07 17:35:22.156615 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.156586 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt"] Jul 07 17:35:22.241425 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.241387 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae6941dc-d3f2-4cf9-bb5d-939837b87997-extensions-socket-volume\") pod \"ae6941dc-d3f2-4cf9-bb5d-939837b87997\" (UID: \"ae6941dc-d3f2-4cf9-bb5d-939837b87997\") " Jul 07 17:35:22.241601 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.241459 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqthf\" (UniqueName: \"kubernetes.io/projected/ae6941dc-d3f2-4cf9-bb5d-939837b87997-kube-api-access-hqthf\") pod \"ae6941dc-d3f2-4cf9-bb5d-939837b87997\" (UID: \"ae6941dc-d3f2-4cf9-bb5d-939837b87997\") " Jul 07 17:35:22.241601 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.241534 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10fcfafe-f5ce-4bed-a64e-901763989900-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xd4lt\" (UID: \"10fcfafe-f5ce-4bed-a64e-901763989900\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" Jul 07 17:35:22.241702 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.241621 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9q9q\" (UniqueName: \"kubernetes.io/projected/10fcfafe-f5ce-4bed-a64e-901763989900-kube-api-access-x9q9q\") pod \"kuadrant-operator-controller-manager-86878c769f-xd4lt\" (UID: \"10fcfafe-f5ce-4bed-a64e-901763989900\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" Jul 07 17:35:22.241834 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.241800 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae6941dc-d3f2-4cf9-bb5d-939837b87997-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ae6941dc-d3f2-4cf9-bb5d-939837b87997" (UID: "ae6941dc-d3f2-4cf9-bb5d-939837b87997"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:35:22.243676 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.243655 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae6941dc-d3f2-4cf9-bb5d-939837b87997-kube-api-access-hqthf" (OuterVolumeSpecName: "kube-api-access-hqthf") pod "ae6941dc-d3f2-4cf9-bb5d-939837b87997" (UID: "ae6941dc-d3f2-4cf9-bb5d-939837b87997"). InnerVolumeSpecName "kube-api-access-hqthf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:35:22.342398 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.342353 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9q9q\" (UniqueName: \"kubernetes.io/projected/10fcfafe-f5ce-4bed-a64e-901763989900-kube-api-access-x9q9q\") pod \"kuadrant-operator-controller-manager-86878c769f-xd4lt\" (UID: \"10fcfafe-f5ce-4bed-a64e-901763989900\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" Jul 07 17:35:22.342398 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.342400 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10fcfafe-f5ce-4bed-a64e-901763989900-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xd4lt\" (UID: \"10fcfafe-f5ce-4bed-a64e-901763989900\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" Jul 07 17:35:22.342617 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.342473 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae6941dc-d3f2-4cf9-bb5d-939837b87997-extensions-socket-volume\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:35:22.342617 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.342483 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqthf\" (UniqueName: \"kubernetes.io/projected/ae6941dc-d3f2-4cf9-bb5d-939837b87997-kube-api-access-hqthf\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:35:22.342779 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.342764 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10fcfafe-f5ce-4bed-a64e-901763989900-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xd4lt\" (UID: \"10fcfafe-f5ce-4bed-a64e-901763989900\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" Jul 07 17:35:22.352635 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.352615 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9q9q\" (UniqueName: \"kubernetes.io/projected/10fcfafe-f5ce-4bed-a64e-901763989900-kube-api-access-x9q9q\") pod \"kuadrant-operator-controller-manager-86878c769f-xd4lt\" (UID: \"10fcfafe-f5ce-4bed-a64e-901763989900\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" Jul 07 17:35:22.453169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.453076 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" Jul 07 17:35:22.598377 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.598343 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt"] Jul 07 17:35:22.602626 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:35:22.602597 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10fcfafe_f5ce_4bed_a64e_901763989900.slice/crio-c6718cfa886ac516bfe1427b65d4ceec6cdc08a54c31d0ced27431580dd77e75 WatchSource:0}: Error finding container c6718cfa886ac516bfe1427b65d4ceec6cdc08a54c31d0ced27431580dd77e75: Status 404 returned error can't find the container with id c6718cfa886ac516bfe1427b65d4ceec6cdc08a54c31d0ced27431580dd77e75 Jul 07 17:35:22.993658 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.993627 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" Jul 07 17:35:22.994140 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.993658 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7" event={"ID":"ae6941dc-d3f2-4cf9-bb5d-939837b87997","Type":"ContainerDied","Data":"570b37072bafe492ad76d0e74df7b12addb816fc184b7c4b70da26ddb3a2e211"} Jul 07 17:35:22.994140 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.993721 2562 scope.go:117] "RemoveContainer" containerID="40ded52f4f54cf0d5d2547cc842018377b7209c6c468edc79f0d02c4e6095fa1" Jul 07 17:35:22.995533 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.995505 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" event={"ID":"10fcfafe-f5ce-4bed-a64e-901763989900","Type":"ContainerStarted","Data":"956285a6b64553ef64463d54caf0fcdd80f8403c592a272f9c92c1cd4c7d5e20"} Jul 07 17:35:22.995686 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.995561 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" event={"ID":"10fcfafe-f5ce-4bed-a64e-901763989900","Type":"ContainerStarted","Data":"c6718cfa886ac516bfe1427b65d4ceec6cdc08a54c31d0ced27431580dd77e75"} Jul 07 17:35:22.995686 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:22.995590 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" Jul 07 17:35:23.031606 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:23.031567 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7"] Jul 07 17:35:23.039406 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:23.039364 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bbb7"] Jul 07 17:35:23.067167 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:23.067113 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" podStartSLOduration=1.067095116 podStartE2EDuration="1.067095116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:35:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:35:23.065732946 +0000 UTC m=+478.947369474" watchObservedRunningTime="2026-07-07 17:35:23.067095116 +0000 UTC m=+478.948731642" Jul 07 17:35:24.637793 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:24.637760 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae6941dc-d3f2-4cf9-bb5d-939837b87997" path="/var/lib/kubelet/pods/ae6941dc-d3f2-4cf9-bb5d-939837b87997/volumes" Jul 07 17:35:29.231710 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:29.231676 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:29.328860 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:29.328827 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:31.689224 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:31.689185 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:31.754137 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:31.754099 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:34.005074 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:34.005042 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" Jul 07 17:35:38.264859 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:38.264820 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:38.336232 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:38.336192 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:41.959011 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:41.958954 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:41.959492 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:41.959455 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f4dt4"] Jul 07 17:35:41.963087 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:41.963071 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" Jul 07 17:35:41.965913 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:41.965894 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 17:35:41.965961 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:41.965899 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4ttxb\"" Jul 07 17:35:41.971004 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:41.970956 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f4dt4"] Jul 07 17:35:42.014627 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.014591 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/71b1293a-e2d9-4f31-ae08-fb33469f75ee-config-file\") pod \"limitador-limitador-64c94fd64f-f4dt4\" (UID: \"71b1293a-e2d9-4f31-ae08-fb33469f75ee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" Jul 07 17:35:42.014796 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.014656 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxqhc\" (UniqueName: \"kubernetes.io/projected/71b1293a-e2d9-4f31-ae08-fb33469f75ee-kube-api-access-fxqhc\") pod \"limitador-limitador-64c94fd64f-f4dt4\" (UID: \"71b1293a-e2d9-4f31-ae08-fb33469f75ee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" Jul 07 17:35:42.059027 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.058990 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f4dt4"] Jul 07 17:35:42.066086 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.066051 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:42.115432 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.115391 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/71b1293a-e2d9-4f31-ae08-fb33469f75ee-config-file\") pod \"limitador-limitador-64c94fd64f-f4dt4\" (UID: \"71b1293a-e2d9-4f31-ae08-fb33469f75ee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" Jul 07 17:35:42.115620 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.115473 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxqhc\" (UniqueName: \"kubernetes.io/projected/71b1293a-e2d9-4f31-ae08-fb33469f75ee-kube-api-access-fxqhc\") pod \"limitador-limitador-64c94fd64f-f4dt4\" (UID: \"71b1293a-e2d9-4f31-ae08-fb33469f75ee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" Jul 07 17:35:42.116085 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.116065 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/71b1293a-e2d9-4f31-ae08-fb33469f75ee-config-file\") pod \"limitador-limitador-64c94fd64f-f4dt4\" (UID: \"71b1293a-e2d9-4f31-ae08-fb33469f75ee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" Jul 07 17:35:42.127549 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.127516 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxqhc\" (UniqueName: \"kubernetes.io/projected/71b1293a-e2d9-4f31-ae08-fb33469f75ee-kube-api-access-fxqhc\") pod \"limitador-limitador-64c94fd64f-f4dt4\" (UID: \"71b1293a-e2d9-4f31-ae08-fb33469f75ee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" Jul 07 17:35:42.130466 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.130435 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:42.274589 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.274494 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" Jul 07 17:35:42.344421 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.344372 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:35:42.347020 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.346999 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-mv272" Jul 07 17:35:42.355564 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.355438 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:35:42.366405 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.366377 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:35:42.417947 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.417912 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c9b353bd-ef2e-4ddb-a232-fb74e2c9b4f6-config-file\") pod \"limitador-limitador-54d6d988d5-mv272\" (UID: \"c9b353bd-ef2e-4ddb-a232-fb74e2c9b4f6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mv272" Jul 07 17:35:42.418129 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.417999 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqzd8\" (UniqueName: \"kubernetes.io/projected/c9b353bd-ef2e-4ddb-a232-fb74e2c9b4f6-kube-api-access-bqzd8\") pod \"limitador-limitador-54d6d988d5-mv272\" (UID: \"c9b353bd-ef2e-4ddb-a232-fb74e2c9b4f6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mv272" Jul 07 17:35:42.424671 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.424645 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f4dt4"] Jul 07 17:35:42.426886 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:35:42.426857 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b1293a_e2d9_4f31_ae08_fb33469f75ee.slice/crio-4d3212e64b4959f1fb8fad3b37bea8a4d46cc95d80edb69baf68461609c85eb1 WatchSource:0}: Error finding container 4d3212e64b4959f1fb8fad3b37bea8a4d46cc95d80edb69baf68461609c85eb1: Status 404 returned error can't find the container with id 4d3212e64b4959f1fb8fad3b37bea8a4d46cc95d80edb69baf68461609c85eb1 Jul 07 17:35:42.518882 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.518848 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c9b353bd-ef2e-4ddb-a232-fb74e2c9b4f6-config-file\") pod \"limitador-limitador-54d6d988d5-mv272\" (UID: \"c9b353bd-ef2e-4ddb-a232-fb74e2c9b4f6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mv272" Jul 07 17:35:42.519104 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.518916 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqzd8\" (UniqueName: \"kubernetes.io/projected/c9b353bd-ef2e-4ddb-a232-fb74e2c9b4f6-kube-api-access-bqzd8\") pod \"limitador-limitador-54d6d988d5-mv272\" (UID: \"c9b353bd-ef2e-4ddb-a232-fb74e2c9b4f6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mv272" Jul 07 17:35:42.519509 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.519489 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c9b353bd-ef2e-4ddb-a232-fb74e2c9b4f6-config-file\") pod \"limitador-limitador-54d6d988d5-mv272\" (UID: \"c9b353bd-ef2e-4ddb-a232-fb74e2c9b4f6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mv272" Jul 07 17:35:42.529014 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.528922 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqzd8\" (UniqueName: \"kubernetes.io/projected/c9b353bd-ef2e-4ddb-a232-fb74e2c9b4f6-kube-api-access-bqzd8\") pod \"limitador-limitador-54d6d988d5-mv272\" (UID: \"c9b353bd-ef2e-4ddb-a232-fb74e2c9b4f6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mv272" Jul 07 17:35:42.660753 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.660718 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-mv272" Jul 07 17:35:42.815285 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.815251 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:35:42.816634 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:35:42.816597 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9b353bd_ef2e_4ddb_a232_fb74e2c9b4f6.slice/crio-e8e5e6b281841aa40af3b78f3ed53554cbe41cebb315af13e794b1f861ac8812 WatchSource:0}: Error finding container e8e5e6b281841aa40af3b78f3ed53554cbe41cebb315af13e794b1f861ac8812: Status 404 returned error can't find the container with id e8e5e6b281841aa40af3b78f3ed53554cbe41cebb315af13e794b1f861ac8812 Jul 07 17:35:42.835775 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.835726 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:42.895886 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:42.895854 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:35:43.090412 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:43.090317 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" event={"ID":"71b1293a-e2d9-4f31-ae08-fb33469f75ee","Type":"ContainerStarted","Data":"4d3212e64b4959f1fb8fad3b37bea8a4d46cc95d80edb69baf68461609c85eb1"} Jul 07 17:35:43.091502 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:43.091478 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-mv272" event={"ID":"c9b353bd-ef2e-4ddb-a232-fb74e2c9b4f6","Type":"ContainerStarted","Data":"e8e5e6b281841aa40af3b78f3ed53554cbe41cebb315af13e794b1f861ac8812"} Jul 07 17:35:46.109452 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:46.109416 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" event={"ID":"71b1293a-e2d9-4f31-ae08-fb33469f75ee","Type":"ContainerStarted","Data":"c7097022d24290ebf8720cce5ce0ba1f951742cf2d2e4a9d850274714de79ea6"} Jul 07 17:35:46.109898 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:46.109509 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" Jul 07 17:35:46.111033 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:46.111009 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-mv272" event={"ID":"c9b353bd-ef2e-4ddb-a232-fb74e2c9b4f6","Type":"ContainerStarted","Data":"20663d21d9351fd58b962e7602d985ee5f920800084a6d7f82a020faf73e4c9d"} Jul 07 17:35:46.111142 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:46.111053 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-mv272" Jul 07 17:35:46.145773 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:46.145720 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-mv272" podStartSLOduration=1.468105166 podStartE2EDuration="4.145703059s" totalImagesPullingTime="2.677597893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:35:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:35:46.145162369 +0000 UTC m=+502.026798898" watchObservedRunningTime="2026-07-07 17:35:46.145703059 +0000 UTC m=+502.027339585" Jul 07 17:35:46.147045 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:46.147015 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" podStartSLOduration=2.086156857 podStartE2EDuration="5.147004071s" totalImagesPullingTime="3.060847214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:35:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:35:46.128148166 +0000 UTC m=+502.009784694" watchObservedRunningTime="2026-07-07 17:35:46.147004071 +0000 UTC m=+502.028640614" Jul 07 17:35:57.117334 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:57.117304 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" Jul 07 17:35:57.117759 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:57.117468 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-mv272" Jul 07 17:35:57.186855 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:57.186821 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f4dt4"] Jul 07 17:35:57.187081 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:57.187033 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" podUID="71b1293a-e2d9-4f31-ae08-fb33469f75ee" containerName="limitador" containerID="cri-o://c7097022d24290ebf8720cce5ce0ba1f951742cf2d2e4a9d850274714de79ea6" gracePeriod=30 Jul 07 17:35:57.739365 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:57.739341 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" Jul 07 17:35:57.856211 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:57.856179 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxqhc\" (UniqueName: \"kubernetes.io/projected/71b1293a-e2d9-4f31-ae08-fb33469f75ee-kube-api-access-fxqhc\") pod \"71b1293a-e2d9-4f31-ae08-fb33469f75ee\" (UID: \"71b1293a-e2d9-4f31-ae08-fb33469f75ee\") " Jul 07 17:35:57.856425 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:57.856324 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/71b1293a-e2d9-4f31-ae08-fb33469f75ee-config-file\") pod \"71b1293a-e2d9-4f31-ae08-fb33469f75ee\" (UID: \"71b1293a-e2d9-4f31-ae08-fb33469f75ee\") " Jul 07 17:35:57.856661 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:57.856636 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71b1293a-e2d9-4f31-ae08-fb33469f75ee-config-file" (OuterVolumeSpecName: "config-file") pod "71b1293a-e2d9-4f31-ae08-fb33469f75ee" (UID: "71b1293a-e2d9-4f31-ae08-fb33469f75ee"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:35:57.858456 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:57.858434 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71b1293a-e2d9-4f31-ae08-fb33469f75ee-kube-api-access-fxqhc" (OuterVolumeSpecName: "kube-api-access-fxqhc") pod "71b1293a-e2d9-4f31-ae08-fb33469f75ee" (UID: "71b1293a-e2d9-4f31-ae08-fb33469f75ee"). InnerVolumeSpecName "kube-api-access-fxqhc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:35:57.957766 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:57.957730 2562 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/71b1293a-e2d9-4f31-ae08-fb33469f75ee-config-file\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:35:57.957766 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:57.957758 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxqhc\" (UniqueName: \"kubernetes.io/projected/71b1293a-e2d9-4f31-ae08-fb33469f75ee-kube-api-access-fxqhc\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:35:58.168993 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:58.168866 2562 generic.go:356] "Generic (PLEG): container finished" podID="71b1293a-e2d9-4f31-ae08-fb33469f75ee" containerID="c7097022d24290ebf8720cce5ce0ba1f951742cf2d2e4a9d850274714de79ea6" exitCode=0 Jul 07 17:35:58.168993 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:58.168922 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" Jul 07 17:35:58.168993 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:58.168931 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" event={"ID":"71b1293a-e2d9-4f31-ae08-fb33469f75ee","Type":"ContainerDied","Data":"c7097022d24290ebf8720cce5ce0ba1f951742cf2d2e4a9d850274714de79ea6"} Jul 07 17:35:58.169562 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:58.169005 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f4dt4" event={"ID":"71b1293a-e2d9-4f31-ae08-fb33469f75ee","Type":"ContainerDied","Data":"4d3212e64b4959f1fb8fad3b37bea8a4d46cc95d80edb69baf68461609c85eb1"} Jul 07 17:35:58.169562 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:58.169030 2562 scope.go:117] "RemoveContainer" containerID="c7097022d24290ebf8720cce5ce0ba1f951742cf2d2e4a9d850274714de79ea6" Jul 07 17:35:58.178456 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:58.178434 2562 scope.go:117] "RemoveContainer" containerID="c7097022d24290ebf8720cce5ce0ba1f951742cf2d2e4a9d850274714de79ea6" Jul 07 17:35:58.178745 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:35:58.178726 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7097022d24290ebf8720cce5ce0ba1f951742cf2d2e4a9d850274714de79ea6\": container with ID starting with c7097022d24290ebf8720cce5ce0ba1f951742cf2d2e4a9d850274714de79ea6 not found: ID does not exist" containerID="c7097022d24290ebf8720cce5ce0ba1f951742cf2d2e4a9d850274714de79ea6" Jul 07 17:35:58.178805 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:58.178755 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7097022d24290ebf8720cce5ce0ba1f951742cf2d2e4a9d850274714de79ea6"} err="failed to get container status \"c7097022d24290ebf8720cce5ce0ba1f951742cf2d2e4a9d850274714de79ea6\": rpc error: code = NotFound desc = could not find container \"c7097022d24290ebf8720cce5ce0ba1f951742cf2d2e4a9d850274714de79ea6\": container with ID starting with c7097022d24290ebf8720cce5ce0ba1f951742cf2d2e4a9d850274714de79ea6 not found: ID does not exist" Jul 07 17:35:58.192140 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:58.192093 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f4dt4"] Jul 07 17:35:58.195110 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:58.195090 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f4dt4"] Jul 07 17:35:58.638107 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:35:58.638074 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71b1293a-e2d9-4f31-ae08-fb33469f75ee" path="/var/lib/kubelet/pods/71b1293a-e2d9-4f31-ae08-fb33469f75ee/volumes" Jul 07 17:36:01.676405 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:01.676368 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-9p5gm"] Jul 07 17:36:01.676935 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:01.676919 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71b1293a-e2d9-4f31-ae08-fb33469f75ee" containerName="limitador" Jul 07 17:36:01.677008 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:01.676938 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="71b1293a-e2d9-4f31-ae08-fb33469f75ee" containerName="limitador" Jul 07 17:36:01.677079 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:01.677066 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="71b1293a-e2d9-4f31-ae08-fb33469f75ee" containerName="limitador" Jul 07 17:36:01.681929 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:01.681904 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-9p5gm" Jul 07 17:36:01.684522 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:01.684495 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-bkzfl\"" Jul 07 17:36:01.684679 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:01.684570 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 07 17:36:01.687542 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:01.687514 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-9p5gm"] Jul 07 17:36:01.710178 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:01.710137 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:01.788587 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:01.788550 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqkdr\" (UniqueName: \"kubernetes.io/projected/3d372ea4-0f57-4f06-847d-f58796d329d3-kube-api-access-cqkdr\") pod \"postgres-69fb4594cf-9p5gm\" (UID: \"3d372ea4-0f57-4f06-847d-f58796d329d3\") " pod="opendatahub/postgres-69fb4594cf-9p5gm" Jul 07 17:36:01.788767 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:01.788622 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3d372ea4-0f57-4f06-847d-f58796d329d3-data\") pod \"postgres-69fb4594cf-9p5gm\" (UID: \"3d372ea4-0f57-4f06-847d-f58796d329d3\") " pod="opendatahub/postgres-69fb4594cf-9p5gm" Jul 07 17:36:01.889669 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:01.889632 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqkdr\" (UniqueName: \"kubernetes.io/projected/3d372ea4-0f57-4f06-847d-f58796d329d3-kube-api-access-cqkdr\") pod \"postgres-69fb4594cf-9p5gm\" (UID: \"3d372ea4-0f57-4f06-847d-f58796d329d3\") " pod="opendatahub/postgres-69fb4594cf-9p5gm" Jul 07 17:36:01.889936 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:01.889699 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3d372ea4-0f57-4f06-847d-f58796d329d3-data\") pod \"postgres-69fb4594cf-9p5gm\" (UID: \"3d372ea4-0f57-4f06-847d-f58796d329d3\") " pod="opendatahub/postgres-69fb4594cf-9p5gm" Jul 07 17:36:01.890068 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:01.890048 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3d372ea4-0f57-4f06-847d-f58796d329d3-data\") pod \"postgres-69fb4594cf-9p5gm\" (UID: \"3d372ea4-0f57-4f06-847d-f58796d329d3\") " pod="opendatahub/postgres-69fb4594cf-9p5gm" Jul 07 17:36:01.900173 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:01.900143 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqkdr\" (UniqueName: \"kubernetes.io/projected/3d372ea4-0f57-4f06-847d-f58796d329d3-kube-api-access-cqkdr\") pod \"postgres-69fb4594cf-9p5gm\" (UID: \"3d372ea4-0f57-4f06-847d-f58796d329d3\") " pod="opendatahub/postgres-69fb4594cf-9p5gm" Jul 07 17:36:01.995444 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:01.995325 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-9p5gm" Jul 07 17:36:02.142374 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:02.142347 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-9p5gm"] Jul 07 17:36:02.145222 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:36:02.145190 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d372ea4_0f57_4f06_847d_f58796d329d3.slice/crio-50b52de9a62c1e01ef06c133d7b1d9097cd70eccd91cf25c4b5b25e15ee5c7bd WatchSource:0}: Error finding container 50b52de9a62c1e01ef06c133d7b1d9097cd70eccd91cf25c4b5b25e15ee5c7bd: Status 404 returned error can't find the container with id 50b52de9a62c1e01ef06c133d7b1d9097cd70eccd91cf25c4b5b25e15ee5c7bd Jul 07 17:36:02.191432 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:02.191396 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-9p5gm" event={"ID":"3d372ea4-0f57-4f06-847d-f58796d329d3","Type":"ContainerStarted","Data":"50b52de9a62c1e01ef06c133d7b1d9097cd70eccd91cf25c4b5b25e15ee5c7bd"} Jul 07 17:36:07.223224 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:07.223189 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-9p5gm" event={"ID":"3d372ea4-0f57-4f06-847d-f58796d329d3","Type":"ContainerStarted","Data":"901afb7ab86951fbc152cb5df33eb0091acf447d3348e5c2089de03f85d43bdc"} Jul 07 17:36:07.223721 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:07.223299 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-9p5gm" Jul 07 17:36:07.246261 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:07.246207 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-9p5gm" podStartSLOduration=1.2991092229999999 podStartE2EDuration="6.246190967s" totalImagesPullingTime="4.947081744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:36:07.244704678 +0000 UTC m=+523.126341206" watchObservedRunningTime="2026-07-07 17:36:07.246190967 +0000 UTC m=+523.127827493" Jul 07 17:36:13.258718 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:13.258687 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-9p5gm" Jul 07 17:36:13.591093 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:13.591051 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:13.872632 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:13.872538 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:20.625824 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:20.625721 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:20.725108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:20.725071 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:20.824613 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:20.824580 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:21.117517 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:21.117481 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:21.719549 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:21.719506 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:22.672629 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:22.672582 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:24.082466 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:24.082434 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:24.177560 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:24.177523 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:51.326649 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:51.326609 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:56.480033 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:56.479954 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:56.976785 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:56.976736 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:57.220250 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:57.220212 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:57.603823 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:57.603781 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:57.702335 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:57.702299 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:36:57.861423 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:36:57.861329 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:37:07.424998 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:07.424943 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:37:07.524282 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:07.524246 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:37:09.250178 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:09.250132 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:37:24.547452 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:24.547427 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 17:37:24.548120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:24.547424 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 17:37:27.333254 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.333216 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq"] Jul 07 17:37:27.338863 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.338804 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.343062 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.343001 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rm7gp\"" Jul 07 17:37:27.343259 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.343051 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 07 17:37:27.343336 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.343074 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 17:37:27.343336 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.343202 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 17:37:27.344015 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.343996 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq"] Jul 07 17:37:27.363131 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.363090 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:37:27.428108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.428073 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.428108 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.428109 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.428347 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.428140 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbrkj\" (UniqueName: \"kubernetes.io/projected/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-kube-api-access-bbrkj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.428347 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.428216 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.428347 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.428290 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.428347 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.428316 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.529144 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.529101 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.529144 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.529147 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.529396 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.529190 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbrkj\" (UniqueName: \"kubernetes.io/projected/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-kube-api-access-bbrkj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.529396 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.529234 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.529396 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.529266 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.529396 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.529294 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.529608 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.529583 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.529653 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.529592 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.529689 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.529639 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.531797 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.531772 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.532109 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.532090 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.543764 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.543728 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbrkj\" (UniqueName: \"kubernetes.io/projected/0070ea1e-c6fd-49ee-b56b-828ab1b0cad2-kube-api-access-bbrkj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq\" (UID: \"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.652281 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.652159 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:27.802709 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.802685 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq"] Jul 07 17:37:27.805001 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:37:27.804959 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0070ea1e_c6fd_49ee_b56b_828ab1b0cad2.slice/crio-8c0794d2f41f7d3ae0a17ba0496e412d67290c4f2c9f4dc5e6dcefb7fc548a2d WatchSource:0}: Error finding container 8c0794d2f41f7d3ae0a17ba0496e412d67290c4f2c9f4dc5e6dcefb7fc548a2d: Status 404 returned error can't find the container with id 8c0794d2f41f7d3ae0a17ba0496e412d67290c4f2c9f4dc5e6dcefb7fc548a2d Jul 07 17:37:27.923361 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:27.923272 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:37:28.645268 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:28.645229 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" event={"ID":"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2","Type":"ContainerStarted","Data":"8c0794d2f41f7d3ae0a17ba0496e412d67290c4f2c9f4dc5e6dcefb7fc548a2d"} Jul 07 17:37:29.651557 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:29.651520 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" event={"ID":"0070ea1e-c6fd-49ee-b56b-828ab1b0cad2","Type":"ContainerStarted","Data":"33a53ab3de3435948cc4203b5a80fc94c71b0a0bf1050003f2566f680d201a52"} Jul 07 17:37:29.671362 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:29.671308 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" podStartSLOduration=1.6151776500000001 podStartE2EDuration="2.671291323s" totalImagesPullingTime="1.056113673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:37:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:37:29.670643425 +0000 UTC m=+605.552279967" watchObservedRunningTime="2026-07-07 17:37:29.671291323 +0000 UTC m=+605.552927849" Jul 07 17:37:37.652877 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:37.652838 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:37.654524 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:37.652899 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:37.665706 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:37.665677 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:37.704304 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:37.704270 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq" Jul 07 17:37:50.565212 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:50.565117 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:37:51.424976 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:51.424921 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:37:53.785203 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:53.785168 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:37:53.848233 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:53.848179 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:37:55.437906 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:55.437872 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:37:56.520392 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:37:56.520354 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:38:03.757025 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:38:03.756958 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:38:05.411207 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:38:05.411158 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:38:15.014691 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:38:15.014647 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:38:17.141682 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:38:17.141648 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:38:25.622860 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:38:25.622777 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:38:28.015991 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:38:28.015941 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:39:24.751785 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:39:24.751687 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:39:24.854084 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:39:24.854049 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:39:26.053438 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:39:26.053401 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:39:26.148914 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:39:26.148884 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:39:34.020531 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:39:34.020498 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:39:34.086726 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:39:34.086689 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:39:57.227274 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:39:57.227237 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:40:06.920067 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:40:06.920030 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:40:17.218627 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:40:17.218588 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:40:26.612887 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:40:26.612843 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:40:37.224697 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:40:37.224655 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:40:39.579493 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:40:39.579443 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:41.206823 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:40:41.206771 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:43.767325 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:40:43.767284 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:43.833701 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:40:43.833662 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:41:04.832299 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:41:04.832262 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:41:17.219451 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:41:17.219408 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:41:21.936410 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:41:21.936372 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:41:22.036645 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:41:22.036609 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:41:22.185522 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:41:22.185483 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:41:28.620874 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:41:28.620832 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:41:28.724492 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:41:28.724456 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:41:28.822251 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:41:28.822208 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:41:28.995818 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:41:28.995698 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:41:35.689712 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:41:35.689678 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:41:35.787800 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:41:35.787762 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:41:53.829535 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:41:53.829499 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:42:03.024574 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:42:03.024538 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:42:12.853474 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:42:12.853436 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:42:21.357567 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:42:21.357487 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:42:21.646234 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:42:21.646142 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:42:24.576906 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:42:24.576874 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 17:42:24.579225 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:42:24.579202 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 17:42:31.128196 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:42:31.128150 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:42:42.267173 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:42:42.267135 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:42:53.851384 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:42:53.851332 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:43:03.960636 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:43:03.960593 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:43:26.128497 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:43:26.128460 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:43:38.829290 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:43:38.829251 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:44:02.620989 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:44:02.620936 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:44:08.666496 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:44:08.666464 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:44:09.756731 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:44:09.756692 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:44:12.920556 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:44:12.920522 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:44:28.523517 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:44:28.523477 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:44:39.717078 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:44:39.717030 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:44:49.026732 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:44:49.026697 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:44:58.410928 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:44:58.410883 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:45:00.143469 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:00.143426 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29724105-kvctv"] Jul 07 17:45:00.147197 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:00.147171 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724105-kvctv" Jul 07 17:45:00.150711 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:00.150691 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-q97km\"" Jul 07 17:45:00.164401 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:00.164374 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724105-kvctv"] Jul 07 17:45:00.252189 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:00.252147 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2st5\" (UniqueName: \"kubernetes.io/projected/081f9a1e-b69b-4ad2-864a-39a22c3110a0-kube-api-access-m2st5\") pod \"maas-api-key-cleanup-29724105-kvctv\" (UID: \"081f9a1e-b69b-4ad2-864a-39a22c3110a0\") " pod="opendatahub/maas-api-key-cleanup-29724105-kvctv" Jul 07 17:45:00.352843 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:00.352803 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2st5\" (UniqueName: \"kubernetes.io/projected/081f9a1e-b69b-4ad2-864a-39a22c3110a0-kube-api-access-m2st5\") pod \"maas-api-key-cleanup-29724105-kvctv\" (UID: \"081f9a1e-b69b-4ad2-864a-39a22c3110a0\") " pod="opendatahub/maas-api-key-cleanup-29724105-kvctv" Jul 07 17:45:00.363042 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:00.363008 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2st5\" (UniqueName: \"kubernetes.io/projected/081f9a1e-b69b-4ad2-864a-39a22c3110a0-kube-api-access-m2st5\") pod \"maas-api-key-cleanup-29724105-kvctv\" (UID: \"081f9a1e-b69b-4ad2-864a-39a22c3110a0\") " pod="opendatahub/maas-api-key-cleanup-29724105-kvctv" Jul 07 17:45:00.458119 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:00.458018 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724105-kvctv" Jul 07 17:45:00.599625 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:00.599594 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724105-kvctv"] Jul 07 17:45:00.601013 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:45:00.600985 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081f9a1e_b69b_4ad2_864a_39a22c3110a0.slice/crio-8c5f7143b8c2102b2f065939d5a0d7d768229439ebb29ee96fa2fed992127850 WatchSource:0}: Error finding container 8c5f7143b8c2102b2f065939d5a0d7d768229439ebb29ee96fa2fed992127850: Status 404 returned error can't find the container with id 8c5f7143b8c2102b2f065939d5a0d7d768229439ebb29ee96fa2fed992127850 Jul 07 17:45:00.602899 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:00.602880 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 17:45:00.856913 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:00.856867 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724105-kvctv" event={"ID":"081f9a1e-b69b-4ad2-864a-39a22c3110a0","Type":"ContainerStarted","Data":"8c5f7143b8c2102b2f065939d5a0d7d768229439ebb29ee96fa2fed992127850"} Jul 07 17:45:03.882782 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:03.882743 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724105-kvctv" event={"ID":"081f9a1e-b69b-4ad2-864a-39a22c3110a0","Type":"ContainerStarted","Data":"c2a0de329105448d9baebe5ac39deb43ba62c1dd7fdabd214820f1aa70c6d4d8"} Jul 07 17:45:03.902261 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:03.902202 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29724105-kvctv" podStartSLOduration=1.559548762 podStartE2EDuration="3.902182254s" totalImagesPullingTime="2.342633492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:45:03.900287273 +0000 UTC m=+1059.781923801" watchObservedRunningTime="2026-07-07 17:45:03.902182254 +0000 UTC m=+1059.783818782" Jul 07 17:45:04.888989 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:04.888944 2562 generic.go:356] "Generic (PLEG): container finished" podID="081f9a1e-b69b-4ad2-864a-39a22c3110a0" containerID="c2a0de329105448d9baebe5ac39deb43ba62c1dd7fdabd214820f1aa70c6d4d8" exitCode=0 Jul 07 17:45:04.889437 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:04.889034 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724105-kvctv" event={"ID":"081f9a1e-b69b-4ad2-864a-39a22c3110a0","Type":"ContainerDied","Data":"c2a0de329105448d9baebe5ac39deb43ba62c1dd7fdabd214820f1aa70c6d4d8"} Jul 07 17:45:06.037626 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:06.037597 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724105-kvctv" Jul 07 17:45:06.102197 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:06.102152 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2st5\" (UniqueName: \"kubernetes.io/projected/081f9a1e-b69b-4ad2-864a-39a22c3110a0-kube-api-access-m2st5\") pod \"081f9a1e-b69b-4ad2-864a-39a22c3110a0\" (UID: \"081f9a1e-b69b-4ad2-864a-39a22c3110a0\") " Jul 07 17:45:06.104569 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:06.104528 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/081f9a1e-b69b-4ad2-864a-39a22c3110a0-kube-api-access-m2st5" (OuterVolumeSpecName: "kube-api-access-m2st5") pod "081f9a1e-b69b-4ad2-864a-39a22c3110a0" (UID: "081f9a1e-b69b-4ad2-864a-39a22c3110a0"). InnerVolumeSpecName "kube-api-access-m2st5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:45:06.202862 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:06.202759 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2st5\" (UniqueName: \"kubernetes.io/projected/081f9a1e-b69b-4ad2-864a-39a22c3110a0-kube-api-access-m2st5\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:45:06.900068 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:06.899975 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724105-kvctv" Jul 07 17:45:06.900068 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:06.900002 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724105-kvctv" event={"ID":"081f9a1e-b69b-4ad2-864a-39a22c3110a0","Type":"ContainerDied","Data":"8c5f7143b8c2102b2f065939d5a0d7d768229439ebb29ee96fa2fed992127850"} Jul 07 17:45:06.900068 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:06.900062 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c5f7143b8c2102b2f065939d5a0d7d768229439ebb29ee96fa2fed992127850" Jul 07 17:45:07.719879 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:07.719839 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:45:33.979502 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:33.979415 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt"] Jul 07 17:45:33.980050 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:33.979630 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" podUID="10fcfafe-f5ce-4bed-a64e-901763989900" containerName="manager" containerID="cri-o://956285a6b64553ef64463d54caf0fcdd80f8403c592a272f9c92c1cd4c7d5e20" gracePeriod=10 Jul 07 17:45:36.677431 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:36.677394 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:45:36.776635 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:36.776598 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:45:37.341279 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:37.341249 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" Jul 07 17:45:37.487341 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:37.487235 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9q9q\" (UniqueName: \"kubernetes.io/projected/10fcfafe-f5ce-4bed-a64e-901763989900-kube-api-access-x9q9q\") pod \"10fcfafe-f5ce-4bed-a64e-901763989900\" (UID: \"10fcfafe-f5ce-4bed-a64e-901763989900\") " Jul 07 17:45:37.487533 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:37.487378 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10fcfafe-f5ce-4bed-a64e-901763989900-extensions-socket-volume\") pod \"10fcfafe-f5ce-4bed-a64e-901763989900\" (UID: \"10fcfafe-f5ce-4bed-a64e-901763989900\") " Jul 07 17:45:37.487761 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:37.487736 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10fcfafe-f5ce-4bed-a64e-901763989900-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "10fcfafe-f5ce-4bed-a64e-901763989900" (UID: "10fcfafe-f5ce-4bed-a64e-901763989900"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:45:37.489418 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:37.489389 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10fcfafe-f5ce-4bed-a64e-901763989900-kube-api-access-x9q9q" (OuterVolumeSpecName: "kube-api-access-x9q9q") pod "10fcfafe-f5ce-4bed-a64e-901763989900" (UID: "10fcfafe-f5ce-4bed-a64e-901763989900"). InnerVolumeSpecName "kube-api-access-x9q9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:45:37.588986 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:37.588927 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10fcfafe-f5ce-4bed-a64e-901763989900-extensions-socket-volume\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:45:37.589162 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:37.588999 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9q9q\" (UniqueName: \"kubernetes.io/projected/10fcfafe-f5ce-4bed-a64e-901763989900-kube-api-access-x9q9q\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:45:38.052599 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:38.052562 2562 generic.go:356] "Generic (PLEG): container finished" podID="10fcfafe-f5ce-4bed-a64e-901763989900" containerID="956285a6b64553ef64463d54caf0fcdd80f8403c592a272f9c92c1cd4c7d5e20" exitCode=0 Jul 07 17:45:38.052599 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:38.052602 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" Jul 07 17:45:38.053139 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:38.052646 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" event={"ID":"10fcfafe-f5ce-4bed-a64e-901763989900","Type":"ContainerDied","Data":"956285a6b64553ef64463d54caf0fcdd80f8403c592a272f9c92c1cd4c7d5e20"} Jul 07 17:45:38.053139 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:38.052701 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt" event={"ID":"10fcfafe-f5ce-4bed-a64e-901763989900","Type":"ContainerDied","Data":"c6718cfa886ac516bfe1427b65d4ceec6cdc08a54c31d0ced27431580dd77e75"} Jul 07 17:45:38.053139 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:38.052728 2562 scope.go:117] "RemoveContainer" containerID="956285a6b64553ef64463d54caf0fcdd80f8403c592a272f9c92c1cd4c7d5e20" Jul 07 17:45:38.064127 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:38.064102 2562 scope.go:117] "RemoveContainer" containerID="956285a6b64553ef64463d54caf0fcdd80f8403c592a272f9c92c1cd4c7d5e20" Jul 07 17:45:38.064496 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:45:38.064474 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"956285a6b64553ef64463d54caf0fcdd80f8403c592a272f9c92c1cd4c7d5e20\": container with ID starting with 956285a6b64553ef64463d54caf0fcdd80f8403c592a272f9c92c1cd4c7d5e20 not found: ID does not exist" containerID="956285a6b64553ef64463d54caf0fcdd80f8403c592a272f9c92c1cd4c7d5e20" Jul 07 17:45:38.064543 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:38.064507 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"956285a6b64553ef64463d54caf0fcdd80f8403c592a272f9c92c1cd4c7d5e20"} err="failed to get container status \"956285a6b64553ef64463d54caf0fcdd80f8403c592a272f9c92c1cd4c7d5e20\": rpc error: code = NotFound desc = could not find container \"956285a6b64553ef64463d54caf0fcdd80f8403c592a272f9c92c1cd4c7d5e20\": container with ID starting with 956285a6b64553ef64463d54caf0fcdd80f8403c592a272f9c92c1cd4c7d5e20 not found: ID does not exist" Jul 07 17:45:38.076852 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:38.076817 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt"] Jul 07 17:45:38.084517 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:38.084488 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xd4lt"] Jul 07 17:45:38.638917 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:45:38.638880 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10fcfafe-f5ce-4bed-a64e-901763989900" path="/var/lib/kubelet/pods/10fcfafe-f5ce-4bed-a64e-901763989900/volumes" Jul 07 17:46:32.319455 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:32.319416 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:46:35.173839 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.173803 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wnwpr"] Jul 07 17:46:35.174274 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.174258 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="081f9a1e-b69b-4ad2-864a-39a22c3110a0" containerName="cleanup" Jul 07 17:46:35.174319 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.174276 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="081f9a1e-b69b-4ad2-864a-39a22c3110a0" containerName="cleanup" Jul 07 17:46:35.174319 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.174294 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10fcfafe-f5ce-4bed-a64e-901763989900" containerName="manager" Jul 07 17:46:35.174319 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.174303 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="10fcfafe-f5ce-4bed-a64e-901763989900" containerName="manager" Jul 07 17:46:35.174408 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.174373 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="10fcfafe-f5ce-4bed-a64e-901763989900" containerName="manager" Jul 07 17:46:35.174408 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.174390 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="081f9a1e-b69b-4ad2-864a-39a22c3110a0" containerName="cleanup" Jul 07 17:46:35.178817 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.178798 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wnwpr" Jul 07 17:46:35.181662 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.181638 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4tcmv\"" Jul 07 17:46:35.189568 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.189542 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wnwpr"] Jul 07 17:46:35.192955 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.192931 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpc4n\" (UniqueName: \"kubernetes.io/projected/d4144f66-b5ed-4c1d-94ec-212faa19b71e-kube-api-access-xpc4n\") pod \"kuadrant-operator-controller-manager-86878c769f-wnwpr\" (UID: \"d4144f66-b5ed-4c1d-94ec-212faa19b71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wnwpr" Jul 07 17:46:35.193109 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.193045 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4144f66-b5ed-4c1d-94ec-212faa19b71e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wnwpr\" (UID: \"d4144f66-b5ed-4c1d-94ec-212faa19b71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wnwpr" Jul 07 17:46:35.293788 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.293724 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4144f66-b5ed-4c1d-94ec-212faa19b71e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wnwpr\" (UID: \"d4144f66-b5ed-4c1d-94ec-212faa19b71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wnwpr" Jul 07 17:46:35.294036 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.293829 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpc4n\" (UniqueName: \"kubernetes.io/projected/d4144f66-b5ed-4c1d-94ec-212faa19b71e-kube-api-access-xpc4n\") pod \"kuadrant-operator-controller-manager-86878c769f-wnwpr\" (UID: \"d4144f66-b5ed-4c1d-94ec-212faa19b71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wnwpr" Jul 07 17:46:35.294169 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.294148 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4144f66-b5ed-4c1d-94ec-212faa19b71e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wnwpr\" (UID: \"d4144f66-b5ed-4c1d-94ec-212faa19b71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wnwpr" Jul 07 17:46:35.304212 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.304190 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpc4n\" (UniqueName: \"kubernetes.io/projected/d4144f66-b5ed-4c1d-94ec-212faa19b71e-kube-api-access-xpc4n\") pod \"kuadrant-operator-controller-manager-86878c769f-wnwpr\" (UID: \"d4144f66-b5ed-4c1d-94ec-212faa19b71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wnwpr" Jul 07 17:46:35.490185 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.490078 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wnwpr" Jul 07 17:46:35.636400 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:35.636377 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wnwpr"] Jul 07 17:46:35.639423 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:46:35.639396 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4144f66_b5ed_4c1d_94ec_212faa19b71e.slice/crio-0d2454cf271c4533f36f55b1ebd2437d67bee58e9572ebb3f0e96bd88669fef5 WatchSource:0}: Error finding container 0d2454cf271c4533f36f55b1ebd2437d67bee58e9572ebb3f0e96bd88669fef5: Status 404 returned error can't find the container with id 0d2454cf271c4533f36f55b1ebd2437d67bee58e9572ebb3f0e96bd88669fef5 Jul 07 17:46:36.336268 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:36.336229 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wnwpr" event={"ID":"d4144f66-b5ed-4c1d-94ec-212faa19b71e","Type":"ContainerStarted","Data":"0da88a3c6ecfc9f557712511ce26a0b15e527edd846ff8ba893154cddf20cb66"} Jul 07 17:46:36.336693 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:36.336291 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wnwpr" event={"ID":"d4144f66-b5ed-4c1d-94ec-212faa19b71e","Type":"ContainerStarted","Data":"0d2454cf271c4533f36f55b1ebd2437d67bee58e9572ebb3f0e96bd88669fef5"} Jul 07 17:46:36.336693 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:36.336430 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wnwpr" Jul 07 17:46:36.359691 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:36.359605 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wnwpr" podStartSLOduration=1.359568702 podStartE2EDuration="1.359568702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:46:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:46:36.358420836 +0000 UTC m=+1152.240057364" watchObservedRunningTime="2026-07-07 17:46:36.359568702 +0000 UTC m=+1152.241205230" Jul 07 17:46:47.343113 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:47.343032 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wnwpr" Jul 07 17:46:55.421332 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:46:55.421293 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:47:06.927207 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:47:06.927171 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:47:15.431317 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:47:15.431283 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:47:24.608882 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:47:24.608850 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 17:47:24.614918 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:47:24.614890 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 17:47:26.271081 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:47:26.271045 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:47:36.518277 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:47:36.518240 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:47:46.623250 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:47:46.623217 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:47:56.222039 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:47:56.222005 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:48:06.855613 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:48:06.855576 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:48:18.549229 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:48:18.549149 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:48:28.237836 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:48:28.237796 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:48:38.531949 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:48:38.531910 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:49:03.137279 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:49:03.137242 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:49:10.322668 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:49:10.322634 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:49:19.724299 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:49:19.724266 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:49:21.357651 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:49:21.357621 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:49:22.595512 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:49:22.595477 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:49:30.223584 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:49:30.223543 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:49:32.348077 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:49:32.348042 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:49:32.444650 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:49:32.444616 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:49:32.544310 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:49:32.544271 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:49:46.829113 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:49:46.829025 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:49:56.420821 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:49:56.420784 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:50:05.417432 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:50:05.417389 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:50:16.339096 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:50:16.339054 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:50:24.931321 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:50:24.931288 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:50:34.527666 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:50:34.527629 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:50:48.425755 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:50:48.425718 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:50:56.915081 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:50:56.915047 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:51:04.828352 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:51:04.828308 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:51:06.321897 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:51:06.321863 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:51:17.130400 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:51:17.130317 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:51:30.216646 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:51:30.216599 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:51:39.312550 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:51:39.312510 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:51:52.022366 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:51:52.022332 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:52:03.417336 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:03.417296 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:52:12.016179 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:12.016143 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:52:15.406983 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:15.406936 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:22.356909 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:22.356869 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:24.646277 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:24.646247 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 17:52:24.647463 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:24.647435 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 17:52:26.222336 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:26.222300 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:52:29.165074 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:29.165035 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:30.657232 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:30.657187 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:35.018666 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:35.018627 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:52:35.860897 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:35.860862 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:38.916930 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:38.916894 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:39.020208 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:39.020164 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:39.437109 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:39.437074 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:43.920196 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:43.920160 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:52:45.926419 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:45.926329 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:45.993314 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:45.993274 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:46.195040 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:46.194933 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:46.266538 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:46.266497 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:47.324510 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:47.324474 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:47.408488 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:47.408450 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:47.489686 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:47.489641 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:47.784165 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:47.784067 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:48.167317 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:48.167282 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:48.267935 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:48.267886 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:50.299983 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:50.299930 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:52.676149 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:52.676105 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:52.925298 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:52.925261 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:53.015665 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:53.015571 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:53.184542 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:53.184498 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:53.925596 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:53.925554 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:52:54.953713 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:54.953629 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:59.279623 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:59.279574 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:59.344335 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:59.344295 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:59.541686 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:59.541579 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:52:59.641118 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:52:59.641083 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:00.675542 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:00.675494 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:00.786417 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:00.786375 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:00.890145 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:00.890106 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:00.988920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:00.988802 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:01.483392 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:01.483324 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:01.580744 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:01.580708 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:02.613862 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:02.613827 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:53:03.042417 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:03.042384 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:06.010219 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:06.010170 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:06.385394 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:06.385358 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:12.285046 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:12.285001 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:12.385131 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:12.385100 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:14.159198 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:14.159153 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:18.796580 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:18.796532 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:18.939322 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:18.939272 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:19.034415 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:19.034379 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:19.102561 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:19.102528 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:20.226996 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:20.226941 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:20.307284 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:20.307244 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:20.388410 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:20.388364 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:20.486037 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:20.485933 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:20.931134 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:20.931078 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:21.030920 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:21.030879 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:22.446956 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:22.446920 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:24.594757 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:24.594715 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:24.787164 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:24.787130 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:24.897123 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:24.897021 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:25.021158 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:25.021118 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:26.954333 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:26.954296 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:31.526860 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:31.526801 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:31.626668 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:31.626632 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:31.888313 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:31.888275 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:31.984454 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:31.984417 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:32.048886 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:32.048828 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:33.515682 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:33.515642 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:39.897933 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:39.897894 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:39.953865 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:39.953825 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:40.078475 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:40.078439 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:40.145261 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:40.145230 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:40.218050 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:40.217931 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:40.322319 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:40.322282 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:40.421209 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:40.421168 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:53:40.834378 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:53:40.834339 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:06.877567 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:06.877527 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:13.493310 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:13.493273 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:13.559280 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:13.559237 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:13.640513 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:13.640479 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:13.743155 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:13.743115 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:29.717798 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:29.717689 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:35.961348 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:35.961310 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:36.031380 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:36.031327 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:36.115078 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:36.115026 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:36.211284 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:36.211251 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:46.765506 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:46.765476 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:47.648641 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:47.648600 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:50.323793 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:50.323751 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:54.566387 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:54.566350 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:54.630863 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:54.630820 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:54.713990 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:54.713943 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:54.814773 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:54.814734 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:54.915150 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:54.915057 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:55.824809 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:55.824759 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:55.922939 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:55.922904 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:56.020139 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:56.020103 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:54:57.223160 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:57.223123 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:54:57.871281 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:54:57.871233 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:04.299769 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:04.299719 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:04.364459 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:04.364425 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:05.061230 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:05.061197 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:05.116795 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:05.116762 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:55:11.527870 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:11.527835 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:11.588306 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:11.588272 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:11.800795 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:11.800690 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:12.143596 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:12.143553 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:12.416602 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:12.416518 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:55:12.460209 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:12.460170 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:15.318380 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:15.318323 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:15.418086 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:15.418047 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:15.590354 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:15.590245 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:16.091433 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:16.091398 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:16.190535 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:16.190499 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:16.291390 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:16.291352 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:19.914915 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:19.914878 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:20.219351 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:20.219248 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:55:20.713391 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:20.713353 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:20.805629 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:20.805595 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:21.221536 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:21.221501 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:27.673190 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:27.673155 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:27.732688 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:27.732656 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:28.241431 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:28.241377 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:28.709721 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:28.709683 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:28.772652 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:28.772608 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:31.917375 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:31.917338 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:55:35.193029 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:35.192994 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:35.259318 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:35.259273 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:35.477435 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:35.477336 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:35.708574 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:35.708537 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:35.832853 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:35.832819 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:35.933495 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:35.933452 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:36.060060 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:36.060010 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:42.641658 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:42.641621 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:42.745545 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:42.745483 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:42.842137 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:42.842095 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:43.035310 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:43.035214 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:49.843120 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:49.843032 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:49.941674 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:49.941636 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:56.920913 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:56.920876 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:57.274045 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:57.273924 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:55:57.847748 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:55:57.847712 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:00.053900 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.053855 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:00.121560 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.121529 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt"] Jul 07 17:56:00.124820 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.124790 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:00.125426 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.125406 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" Jul 07 17:56:00.128342 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.128302 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-8a43fdf7-serving-cert\"" Jul 07 17:56:00.128342 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.128324 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-q97km\"" Jul 07 17:56:00.135773 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.135744 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ace1de89-8428-4902-bb79-91f67e5eba8c-maas-api-tls\") pod \"maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt\" (UID: \"ace1de89-8428-4902-bb79-91f67e5eba8c\") " pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" Jul 07 17:56:00.135892 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.135787 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8dvt\" (UniqueName: \"kubernetes.io/projected/ace1de89-8428-4902-bb79-91f67e5eba8c-kube-api-access-x8dvt\") pod \"maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt\" (UID: \"ace1de89-8428-4902-bb79-91f67e5eba8c\") " pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" Jul 07 17:56:00.136047 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.136007 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ace1de89-8428-4902-bb79-91f67e5eba8c-openshift-service-ca\") pod \"maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt\" (UID: \"ace1de89-8428-4902-bb79-91f67e5eba8c\") " pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" Jul 07 17:56:00.136131 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.136086 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt"] Jul 07 17:56:00.192622 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.192588 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:00.236702 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.236659 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ace1de89-8428-4902-bb79-91f67e5eba8c-openshift-service-ca\") pod \"maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt\" (UID: \"ace1de89-8428-4902-bb79-91f67e5eba8c\") " pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" Jul 07 17:56:00.236894 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.236718 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ace1de89-8428-4902-bb79-91f67e5eba8c-maas-api-tls\") pod \"maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt\" (UID: \"ace1de89-8428-4902-bb79-91f67e5eba8c\") " pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" Jul 07 17:56:00.236894 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.236739 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8dvt\" (UniqueName: \"kubernetes.io/projected/ace1de89-8428-4902-bb79-91f67e5eba8c-kube-api-access-x8dvt\") pod \"maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt\" (UID: \"ace1de89-8428-4902-bb79-91f67e5eba8c\") " pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" Jul 07 17:56:00.236894 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:56:00.236875 2562 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-8a43fdf7-serving-cert: secret "maas-api-e2e-mt-8a43fdf7-serving-cert" not found Jul 07 17:56:00.237072 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:56:00.236956 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ace1de89-8428-4902-bb79-91f67e5eba8c-maas-api-tls podName:ace1de89-8428-4902-bb79-91f67e5eba8c nodeName:}" failed. No retries permitted until 2026-07-07 17:56:00.736934915 +0000 UTC m=+1716.618571420 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ace1de89-8428-4902-bb79-91f67e5eba8c-maas-api-tls") pod "maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" (UID: "ace1de89-8428-4902-bb79-91f67e5eba8c") : secret "maas-api-e2e-mt-8a43fdf7-serving-cert" not found Jul 07 17:56:00.237386 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.237364 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ace1de89-8428-4902-bb79-91f67e5eba8c-openshift-service-ca\") pod \"maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt\" (UID: \"ace1de89-8428-4902-bb79-91f67e5eba8c\") " pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" Jul 07 17:56:00.252077 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.252045 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8dvt\" (UniqueName: \"kubernetes.io/projected/ace1de89-8428-4902-bb79-91f67e5eba8c-kube-api-access-x8dvt\") pod \"maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt\" (UID: \"ace1de89-8428-4902-bb79-91f67e5eba8c\") " pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" Jul 07 17:56:00.408099 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.408058 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:00.672818 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.672723 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:00.740073 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.740030 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ace1de89-8428-4902-bb79-91f67e5eba8c-maas-api-tls\") pod \"maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt\" (UID: \"ace1de89-8428-4902-bb79-91f67e5eba8c\") " pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" Jul 07 17:56:00.742665 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.742645 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ace1de89-8428-4902-bb79-91f67e5eba8c-maas-api-tls\") pod \"maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt\" (UID: \"ace1de89-8428-4902-bb79-91f67e5eba8c\") " pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" Jul 07 17:56:00.769952 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:00.769921 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:01.037489 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:01.037393 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" Jul 07 17:56:01.182595 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:01.182570 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt"] Jul 07 17:56:01.185653 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:56:01.185617 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podace1de89_8428_4902_bb79_91f67e5eba8c.slice/crio-9ec10bc2b6925565a1cebd382b2bccea62a816d33b59777de6f8ced553434e79 WatchSource:0}: Error finding container 9ec10bc2b6925565a1cebd382b2bccea62a816d33b59777de6f8ced553434e79: Status 404 returned error can't find the container with id 9ec10bc2b6925565a1cebd382b2bccea62a816d33b59777de6f8ced553434e79 Jul 07 17:56:01.187068 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:01.187048 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 17:56:02.118666 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:02.118626 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" event={"ID":"ace1de89-8428-4902-bb79-91f67e5eba8c","Type":"ContainerStarted","Data":"9ec10bc2b6925565a1cebd382b2bccea62a816d33b59777de6f8ced553434e79"} Jul 07 17:56:04.016307 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:04.016273 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:04.117170 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:04.117130 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:04.130501 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:04.130463 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" event={"ID":"ace1de89-8428-4902-bb79-91f67e5eba8c","Type":"ContainerStarted","Data":"d2a3f896374003552ab1734e759cea344cb740b7d5c85328305c59d611f1ee53"} Jul 07 17:56:04.130671 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:04.130599 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" Jul 07 17:56:04.152534 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:04.152485 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" podStartSLOduration=1.865374993 podStartE2EDuration="4.152468639s" totalImagesPullingTime="2.287093646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:56:04.150814537 +0000 UTC m=+1720.032451064" watchObservedRunningTime="2026-07-07 17:56:04.152468639 +0000 UTC m=+1720.034105166" Jul 07 17:56:04.585015 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:04.584953 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:07.215993 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:07.215940 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:56:10.141962 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:10.141932 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" Jul 07 17:56:18.645875 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:18.645842 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:21.460873 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:21.460829 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:21.562151 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:21.562113 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:21.926585 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:21.926549 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:56:36.519478 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:36.519438 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:56:42.487843 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:42.487806 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:42.939344 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:42.939303 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt"] Jul 07 17:56:42.939568 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:42.939545 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" podUID="ace1de89-8428-4902-bb79-91f67e5eba8c" containerName="maas-api" containerID="cri-o://d2a3f896374003552ab1734e759cea344cb740b7d5c85328305c59d611f1ee53" gracePeriod=30 Jul 07 17:56:42.959466 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:42.959436 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:43.197655 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.197597 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" Jul 07 17:56:43.327501 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.327467 2562 generic.go:356] "Generic (PLEG): container finished" podID="ace1de89-8428-4902-bb79-91f67e5eba8c" containerID="d2a3f896374003552ab1734e759cea344cb740b7d5c85328305c59d611f1ee53" exitCode=0 Jul 07 17:56:43.327665 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.327512 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" Jul 07 17:56:43.327665 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.327515 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" event={"ID":"ace1de89-8428-4902-bb79-91f67e5eba8c","Type":"ContainerDied","Data":"d2a3f896374003552ab1734e759cea344cb740b7d5c85328305c59d611f1ee53"} Jul 07 17:56:43.327665 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.327573 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt" event={"ID":"ace1de89-8428-4902-bb79-91f67e5eba8c","Type":"ContainerDied","Data":"9ec10bc2b6925565a1cebd382b2bccea62a816d33b59777de6f8ced553434e79"} Jul 07 17:56:43.327665 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.327597 2562 scope.go:117] "RemoveContainer" containerID="d2a3f896374003552ab1734e759cea344cb740b7d5c85328305c59d611f1ee53" Jul 07 17:56:43.328028 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.328009 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ace1de89-8428-4902-bb79-91f67e5eba8c-openshift-service-ca\") pod \"ace1de89-8428-4902-bb79-91f67e5eba8c\" (UID: \"ace1de89-8428-4902-bb79-91f67e5eba8c\") " Jul 07 17:56:43.328137 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.328117 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8dvt\" (UniqueName: \"kubernetes.io/projected/ace1de89-8428-4902-bb79-91f67e5eba8c-kube-api-access-x8dvt\") pod \"ace1de89-8428-4902-bb79-91f67e5eba8c\" (UID: \"ace1de89-8428-4902-bb79-91f67e5eba8c\") " Jul 07 17:56:43.328202 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.328169 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ace1de89-8428-4902-bb79-91f67e5eba8c-maas-api-tls\") pod \"ace1de89-8428-4902-bb79-91f67e5eba8c\" (UID: \"ace1de89-8428-4902-bb79-91f67e5eba8c\") " Jul 07 17:56:43.328449 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.328401 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ace1de89-8428-4902-bb79-91f67e5eba8c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ace1de89-8428-4902-bb79-91f67e5eba8c" (UID: "ace1de89-8428-4902-bb79-91f67e5eba8c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:56:43.330492 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.330456 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ace1de89-8428-4902-bb79-91f67e5eba8c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ace1de89-8428-4902-bb79-91f67e5eba8c" (UID: "ace1de89-8428-4902-bb79-91f67e5eba8c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:56:43.330571 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.330536 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ace1de89-8428-4902-bb79-91f67e5eba8c-kube-api-access-x8dvt" (OuterVolumeSpecName: "kube-api-access-x8dvt") pod "ace1de89-8428-4902-bb79-91f67e5eba8c" (UID: "ace1de89-8428-4902-bb79-91f67e5eba8c"). InnerVolumeSpecName "kube-api-access-x8dvt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:56:43.348159 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.348137 2562 scope.go:117] "RemoveContainer" containerID="d2a3f896374003552ab1734e759cea344cb740b7d5c85328305c59d611f1ee53" Jul 07 17:56:43.348442 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:56:43.348419 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2a3f896374003552ab1734e759cea344cb740b7d5c85328305c59d611f1ee53\": container with ID starting with d2a3f896374003552ab1734e759cea344cb740b7d5c85328305c59d611f1ee53 not found: ID does not exist" containerID="d2a3f896374003552ab1734e759cea344cb740b7d5c85328305c59d611f1ee53" Jul 07 17:56:43.348497 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.348451 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2a3f896374003552ab1734e759cea344cb740b7d5c85328305c59d611f1ee53"} err="failed to get container status \"d2a3f896374003552ab1734e759cea344cb740b7d5c85328305c59d611f1ee53\": rpc error: code = NotFound desc = could not find container \"d2a3f896374003552ab1734e759cea344cb740b7d5c85328305c59d611f1ee53\": container with ID starting with d2a3f896374003552ab1734e759cea344cb740b7d5c85328305c59d611f1ee53 not found: ID does not exist" Jul 07 17:56:43.429079 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.429042 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8dvt\" (UniqueName: \"kubernetes.io/projected/ace1de89-8428-4902-bb79-91f67e5eba8c-kube-api-access-x8dvt\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:56:43.429079 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.429074 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ace1de89-8428-4902-bb79-91f67e5eba8c-maas-api-tls\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:56:43.429079 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.429087 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ace1de89-8428-4902-bb79-91f67e5eba8c-openshift-service-ca\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 17:56:43.652237 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.652198 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt"] Jul 07 17:56:43.656992 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:43.656938 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-8a43fdf7-d79cb4b7f-j72dt"] Jul 07 17:56:44.637681 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:44.637641 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ace1de89-8428-4902-bb79-91f67e5eba8c" path="/var/lib/kubelet/pods/ace1de89-8428-4902-bb79-91f67e5eba8c/volumes" Jul 07 17:56:46.741826 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:46.741787 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:56:48.665927 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:48.665877 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:53.700665 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:53.700608 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:56:58.120739 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:56:58.120704 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:57:24.674450 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:24.674418 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 17:57:24.676776 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:24.676755 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 17:57:29.924203 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:29.924165 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:31.376273 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:31.376231 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:33.212861 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:33.212767 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:57:37.535556 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:37.535521 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:37.637113 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:37.637079 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:37.897603 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:37.897567 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:38.279121 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:38.279011 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:39.410416 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:39.410376 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:39.509245 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:39.509205 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:39.609053 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:39.609012 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:40.460944 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:40.460903 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:40.546650 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:40.546591 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:40.645835 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:40.645800 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:42.104305 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:42.104268 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:42.312386 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:42.312349 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:57:48.556106 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:48.556065 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:48.656476 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:48.656435 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:49.296259 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:49.296225 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:53.422037 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:53.421994 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:57:55.495139 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:55.495096 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:55.602511 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:55.602465 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:57:55.700873 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:57:55.700830 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:06.431014 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:06.430953 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:06.901958 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:06.901919 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:11.611560 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:11.611522 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:24.075847 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:24.075805 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:24.135342 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:24.135311 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:28.152181 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:28.152148 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:34.487179 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:34.487145 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:34.554438 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:34.554402 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:34.982638 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:34.982600 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:35.487091 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:35.487054 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:35.585760 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:35.585726 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:39.076488 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.076451 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:39.175473 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.175434 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:39.320907 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.320851 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf"] Jul 07 17:58:39.321429 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.321405 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ace1de89-8428-4902-bb79-91f67e5eba8c" containerName="maas-api" Jul 07 17:58:39.321597 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.321582 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ace1de89-8428-4902-bb79-91f67e5eba8c" containerName="maas-api" Jul 07 17:58:39.321772 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.321754 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ace1de89-8428-4902-bb79-91f67e5eba8c" containerName="maas-api" Jul 07 17:58:39.324922 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.324902 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" Jul 07 17:58:39.327769 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.327714 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-c65a1b-serving-cert\"" Jul 07 17:58:39.327945 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.327923 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-q97km\"" Jul 07 17:58:39.336192 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.336169 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf"] Jul 07 17:58:39.352903 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.352873 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf\" (UID: \"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54\") " pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" Jul 07 17:58:39.353146 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.352923 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-maas-api-tls\") pod \"maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf\" (UID: \"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54\") " pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" Jul 07 17:58:39.353146 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.352997 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8q6k\" (UniqueName: \"kubernetes.io/projected/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-kube-api-access-q8q6k\") pod \"maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf\" (UID: \"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54\") " pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" Jul 07 17:58:39.366589 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.366555 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:39.453758 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.453724 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf\" (UID: \"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54\") " pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" Jul 07 17:58:39.453758 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.453765 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-maas-api-tls\") pod \"maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf\" (UID: \"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54\") " pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" Jul 07 17:58:39.454008 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.453804 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8q6k\" (UniqueName: \"kubernetes.io/projected/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-kube-api-access-q8q6k\") pod \"maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf\" (UID: \"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54\") " pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" Jul 07 17:58:39.454008 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:58:39.453919 2562 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-c65a1b-serving-cert: secret "maas-api-e2e-inf-a-c65a1b-serving-cert" not found Jul 07 17:58:39.454089 ip-10-0-141-75 kubenswrapper[2562]: E0707 17:58:39.454013 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-maas-api-tls podName:cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54 nodeName:}" failed. No retries permitted until 2026-07-07 17:58:39.953992872 +0000 UTC m=+1875.835629399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-maas-api-tls") pod "maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" (UID: "cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54") : secret "maas-api-e2e-inf-a-c65a1b-serving-cert" not found Jul 07 17:58:39.454489 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.454470 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf\" (UID: \"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54\") " pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" Jul 07 17:58:39.463635 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.463607 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8q6k\" (UniqueName: \"kubernetes.io/projected/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-kube-api-access-q8q6k\") pod \"maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf\" (UID: \"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54\") " pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" Jul 07 17:58:39.534567 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.534528 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:39.897285 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.897253 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:39.958476 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.958430 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-maas-api-tls\") pod \"maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf\" (UID: \"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54\") " pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" Jul 07 17:58:39.961151 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.961122 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-maas-api-tls\") pod \"maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf\" (UID: \"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54\") " pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" Jul 07 17:58:39.993110 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:39.993058 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:40.237547 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:40.237446 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" Jul 07 17:58:40.383092 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:40.383063 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf"] Jul 07 17:58:40.385239 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:58:40.385210 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfe0f9a9_a48f_4b7e_bfdf_ffe49dab7c54.slice/crio-aee106e74948a6713e806565665dd7686e8d5934bdb0929214ad37f235be2acc WatchSource:0}: Error finding container aee106e74948a6713e806565665dd7686e8d5934bdb0929214ad37f235be2acc: Status 404 returned error can't find the container with id aee106e74948a6713e806565665dd7686e8d5934bdb0929214ad37f235be2acc Jul 07 17:58:40.902678 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:40.902648 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" event={"ID":"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54","Type":"ContainerStarted","Data":"bc34093fcf08b0e9343a0f4b4e979793e6e05c4b3652e0c7036a7de24d88e5e9"} Jul 07 17:58:40.902837 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:40.902697 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" event={"ID":"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54","Type":"ContainerStarted","Data":"aee106e74948a6713e806565665dd7686e8d5934bdb0929214ad37f235be2acc"} Jul 07 17:58:40.902837 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:40.902752 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" Jul 07 17:58:40.924321 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:40.924271 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" podStartSLOduration=1.492229688 podStartE2EDuration="1.924253482s" totalImagesPullingTime="432.023794ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:58:40.921250934 +0000 UTC m=+1876.802887464" watchObservedRunningTime="2026-07-07 17:58:40.924253482 +0000 UTC m=+1876.805890008" Jul 07 17:58:43.680278 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:43.680239 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:44.064661 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:44.064625 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:44.398788 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:44.398693 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:45.118021 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:45.117961 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:46.823552 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:46.823514 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:58:46.913017 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:46.912992 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" Jul 07 17:58:46.935674 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:46.935637 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:47.035598 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.035562 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:47.129409 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.129316 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz"] Jul 07 17:58:47.133173 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.133147 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" Jul 07 17:58:47.136140 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.136115 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-28127c-serving-cert\"" Jul 07 17:58:47.147061 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.147034 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz"] Jul 07 17:58:47.177450 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.177417 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:47.211404 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.211371 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da56bd28-4170-49cf-997c-84817ff81a8b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-28127c-f64c6df78-xflfz\" (UID: \"da56bd28-4170-49cf-997c-84817ff81a8b\") " pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" Jul 07 17:58:47.211583 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.211437 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da56bd28-4170-49cf-997c-84817ff81a8b-maas-api-tls\") pod \"maas-api-e2e-inf-b-28127c-f64c6df78-xflfz\" (UID: \"da56bd28-4170-49cf-997c-84817ff81a8b\") " pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" Jul 07 17:58:47.211583 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.211538 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8wkx\" (UniqueName: \"kubernetes.io/projected/da56bd28-4170-49cf-997c-84817ff81a8b-kube-api-access-t8wkx\") pod \"maas-api-e2e-inf-b-28127c-f64c6df78-xflfz\" (UID: \"da56bd28-4170-49cf-997c-84817ff81a8b\") " pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" Jul 07 17:58:47.249735 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.249702 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:47.312562 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.312524 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da56bd28-4170-49cf-997c-84817ff81a8b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-28127c-f64c6df78-xflfz\" (UID: \"da56bd28-4170-49cf-997c-84817ff81a8b\") " pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" Jul 07 17:58:47.312762 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.312584 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da56bd28-4170-49cf-997c-84817ff81a8b-maas-api-tls\") pod \"maas-api-e2e-inf-b-28127c-f64c6df78-xflfz\" (UID: \"da56bd28-4170-49cf-997c-84817ff81a8b\") " pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" Jul 07 17:58:47.312762 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.312611 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8wkx\" (UniqueName: \"kubernetes.io/projected/da56bd28-4170-49cf-997c-84817ff81a8b-kube-api-access-t8wkx\") pod \"maas-api-e2e-inf-b-28127c-f64c6df78-xflfz\" (UID: \"da56bd28-4170-49cf-997c-84817ff81a8b\") " pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" Jul 07 17:58:47.313225 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.313201 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da56bd28-4170-49cf-997c-84817ff81a8b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-28127c-f64c6df78-xflfz\" (UID: \"da56bd28-4170-49cf-997c-84817ff81a8b\") " pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" Jul 07 17:58:47.315124 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.315099 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da56bd28-4170-49cf-997c-84817ff81a8b-maas-api-tls\") pod \"maas-api-e2e-inf-b-28127c-f64c6df78-xflfz\" (UID: \"da56bd28-4170-49cf-997c-84817ff81a8b\") " pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" Jul 07 17:58:47.322586 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.322555 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8wkx\" (UniqueName: \"kubernetes.io/projected/da56bd28-4170-49cf-997c-84817ff81a8b-kube-api-access-t8wkx\") pod \"maas-api-e2e-inf-b-28127c-f64c6df78-xflfz\" (UID: \"da56bd28-4170-49cf-997c-84817ff81a8b\") " pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" Jul 07 17:58:47.447739 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.447644 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" Jul 07 17:58:47.590233 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.590203 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz"] Jul 07 17:58:47.592195 ip-10-0-141-75 kubenswrapper[2562]: W0707 17:58:47.592163 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda56bd28_4170_49cf_997c_84817ff81a8b.slice/crio-0dd75832cd5f99982eb3a6af4d2366fa8b5bb49acee4c42346b028f76fc699e5 WatchSource:0}: Error finding container 0dd75832cd5f99982eb3a6af4d2366fa8b5bb49acee4c42346b028f76fc699e5: Status 404 returned error can't find the container with id 0dd75832cd5f99982eb3a6af4d2366fa8b5bb49acee4c42346b028f76fc699e5 Jul 07 17:58:47.677574 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.677519 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:47.778260 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.778165 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:47.939896 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:47.939861 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" event={"ID":"da56bd28-4170-49cf-997c-84817ff81a8b","Type":"ContainerStarted","Data":"0dd75832cd5f99982eb3a6af4d2366fa8b5bb49acee4c42346b028f76fc699e5"} Jul 07 17:58:48.945861 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:48.945826 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" event={"ID":"da56bd28-4170-49cf-997c-84817ff81a8b","Type":"ContainerStarted","Data":"d448990b891decd62be14bff86a82fecfeeeaae21f9dc898bda5d39893076b7d"} Jul 07 17:58:48.946288 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:48.945908 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" Jul 07 17:58:48.965926 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:48.965869 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" podStartSLOduration=1.503198474 podStartE2EDuration="1.965847195s" totalImagesPullingTime="462.648721ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:58:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:58:48.965077685 +0000 UTC m=+1884.846714222" watchObservedRunningTime="2026-07-07 17:58:48.965847195 +0000 UTC m=+1884.847483727" Jul 07 17:58:51.537689 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:51.537652 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:58:54.957038 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:54.957006 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" Jul 07 17:58:57.708814 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:58:57.708777 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:59:00.717724 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:59:00.717686 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:59:00.792612 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:59:00.792573 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:59:09.645657 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:59:09.645626 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:59:09.746668 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:59:09.746604 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:59:15.407723 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:59:15.407689 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:59:18.812206 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:59:18.812165 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:59:29.417744 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:59:29.417703 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 17:59:53.228408 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:59:53.228363 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 17:59:58.366952 ip-10-0-141-75 kubenswrapper[2562]: I0707 17:59:58.366909 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:00:00.140664 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.140622 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j"] Jul 07 18:00:00.144622 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.144593 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29724120-kpbdw"] Jul 07 18:00:00.144793 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.144737 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j" Jul 07 18:00:00.148245 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.148225 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct"] Jul 07 18:00:00.148381 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.148343 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724120-kpbdw" Jul 07 18:00:00.151323 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.151290 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct" Jul 07 18:00:00.153928 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.153869 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j"] Jul 07 18:00:00.157923 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.157864 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724120-kpbdw"] Jul 07 18:00:00.161880 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.160769 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct"] Jul 07 18:00:00.241537 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.241507 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5gz5\" (UniqueName: \"kubernetes.io/projected/3f43826b-2206-4855-8553-ca53cc16778c-kube-api-access-s5gz5\") pod \"maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct\" (UID: \"3f43826b-2206-4855-8553-ca53cc16778c\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct" Jul 07 18:00:00.241701 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.241558 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m5vs\" (UniqueName: \"kubernetes.io/projected/33d6d987-77e9-4b1b-ba76-e3237eadcb15-kube-api-access-8m5vs\") pod \"maas-api-key-cleanup-29724120-kpbdw\" (UID: \"33d6d987-77e9-4b1b-ba76-e3237eadcb15\") " pod="opendatahub/maas-api-key-cleanup-29724120-kpbdw" Jul 07 18:00:00.241701 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.241618 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm9wz\" (UniqueName: \"kubernetes.io/projected/151d029c-c12c-43f4-9d35-8304a02ab5c5-kube-api-access-fm9wz\") pod \"maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j\" (UID: \"151d029c-c12c-43f4-9d35-8304a02ab5c5\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j" Jul 07 18:00:00.342962 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.342923 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5gz5\" (UniqueName: \"kubernetes.io/projected/3f43826b-2206-4855-8553-ca53cc16778c-kube-api-access-s5gz5\") pod \"maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct\" (UID: \"3f43826b-2206-4855-8553-ca53cc16778c\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct" Jul 07 18:00:00.343164 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.343020 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m5vs\" (UniqueName: \"kubernetes.io/projected/33d6d987-77e9-4b1b-ba76-e3237eadcb15-kube-api-access-8m5vs\") pod \"maas-api-key-cleanup-29724120-kpbdw\" (UID: \"33d6d987-77e9-4b1b-ba76-e3237eadcb15\") " pod="opendatahub/maas-api-key-cleanup-29724120-kpbdw" Jul 07 18:00:00.343164 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.343054 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9wz\" (UniqueName: \"kubernetes.io/projected/151d029c-c12c-43f4-9d35-8304a02ab5c5-kube-api-access-fm9wz\") pod \"maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j\" (UID: \"151d029c-c12c-43f4-9d35-8304a02ab5c5\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j" Jul 07 18:00:00.354076 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.354049 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m5vs\" (UniqueName: \"kubernetes.io/projected/33d6d987-77e9-4b1b-ba76-e3237eadcb15-kube-api-access-8m5vs\") pod \"maas-api-key-cleanup-29724120-kpbdw\" (UID: \"33d6d987-77e9-4b1b-ba76-e3237eadcb15\") " pod="opendatahub/maas-api-key-cleanup-29724120-kpbdw" Jul 07 18:00:00.354218 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.354180 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5gz5\" (UniqueName: \"kubernetes.io/projected/3f43826b-2206-4855-8553-ca53cc16778c-kube-api-access-s5gz5\") pod \"maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct\" (UID: \"3f43826b-2206-4855-8553-ca53cc16778c\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct" Jul 07 18:00:00.354260 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.354226 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm9wz\" (UniqueName: \"kubernetes.io/projected/151d029c-c12c-43f4-9d35-8304a02ab5c5-kube-api-access-fm9wz\") pod \"maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j\" (UID: \"151d029c-c12c-43f4-9d35-8304a02ab5c5\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j" Jul 07 18:00:00.460320 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.460223 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j" Jul 07 18:00:00.470627 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.470598 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724120-kpbdw" Jul 07 18:00:00.477717 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.477691 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct" Jul 07 18:00:00.635445 ip-10-0-141-75 kubenswrapper[2562]: W0707 18:00:00.635413 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod151d029c_c12c_43f4_9d35_8304a02ab5c5.slice/crio-8eb977cd8ea066abe7ddc3533d36f49e8472661bb0e5545881bc7d04aba2f742 WatchSource:0}: Error finding container 8eb977cd8ea066abe7ddc3533d36f49e8472661bb0e5545881bc7d04aba2f742: Status 404 returned error can't find the container with id 8eb977cd8ea066abe7ddc3533d36f49e8472661bb0e5545881bc7d04aba2f742 Jul 07 18:00:00.646638 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.646589 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j"] Jul 07 18:00:00.873433 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.873268 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724120-kpbdw"] Jul 07 18:00:00.875951 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:00.875921 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct"] Jul 07 18:00:00.876086 ip-10-0-141-75 kubenswrapper[2562]: W0707 18:00:00.876037 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f43826b_2206_4855_8553_ca53cc16778c.slice/crio-6923714a93c48f4e00d295d397ef7c39a3129329c298e5214f0243059c40c321 WatchSource:0}: Error finding container 6923714a93c48f4e00d295d397ef7c39a3129329c298e5214f0243059c40c321: Status 404 returned error can't find the container with id 6923714a93c48f4e00d295d397ef7c39a3129329c298e5214f0243059c40c321 Jul 07 18:00:00.877071 ip-10-0-141-75 kubenswrapper[2562]: W0707 18:00:00.877045 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d6d987_77e9_4b1b_ba76_e3237eadcb15.slice/crio-a3cf6ca3dbdfe4319c3d90b2f9050d13e985bbba085adb7a4ffc23ad01f72ff5 WatchSource:0}: Error finding container a3cf6ca3dbdfe4319c3d90b2f9050d13e985bbba085adb7a4ffc23ad01f72ff5: Status 404 returned error can't find the container with id a3cf6ca3dbdfe4319c3d90b2f9050d13e985bbba085adb7a4ffc23ad01f72ff5 Jul 07 18:00:01.314066 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:01.314033 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct" event={"ID":"3f43826b-2206-4855-8553-ca53cc16778c","Type":"ContainerStarted","Data":"26033f4332b15d55a93bb655121a7e2662b2926124d7ad1f756b03a924c4b29f"} Jul 07 18:00:01.314496 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:01.314089 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct" event={"ID":"3f43826b-2206-4855-8553-ca53cc16778c","Type":"ContainerStarted","Data":"6923714a93c48f4e00d295d397ef7c39a3129329c298e5214f0243059c40c321"} Jul 07 18:00:01.315676 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:01.315647 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724120-kpbdw" event={"ID":"33d6d987-77e9-4b1b-ba76-e3237eadcb15","Type":"ContainerStarted","Data":"69baf8f7852a7a6f3e5e191a376c77c02bb20d2687de330d39dab4887ab47fc0"} Jul 07 18:00:01.315826 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:01.315689 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724120-kpbdw" event={"ID":"33d6d987-77e9-4b1b-ba76-e3237eadcb15","Type":"ContainerStarted","Data":"a3cf6ca3dbdfe4319c3d90b2f9050d13e985bbba085adb7a4ffc23ad01f72ff5"} Jul 07 18:00:01.317195 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:01.317160 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j" event={"ID":"151d029c-c12c-43f4-9d35-8304a02ab5c5","Type":"ContainerStarted","Data":"da938af4f64f965e70aad84703bc812b0ac6bfeba493cfcb73ca8eea667cfccd"} Jul 07 18:00:01.317322 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:01.317199 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j" event={"ID":"151d029c-c12c-43f4-9d35-8304a02ab5c5","Type":"ContainerStarted","Data":"8eb977cd8ea066abe7ddc3533d36f49e8472661bb0e5545881bc7d04aba2f742"} Jul 07 18:00:01.337112 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:01.337050 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct" podStartSLOduration=1.337030091 podStartE2EDuration="1.337030091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:00:01.334950085 +0000 UTC m=+1957.216586608" watchObservedRunningTime="2026-07-07 18:00:01.337030091 +0000 UTC m=+1957.218666619" Jul 07 18:00:01.355741 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:01.355678 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j" podStartSLOduration=1.355657214 podStartE2EDuration="1.355657214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:00:01.353236945 +0000 UTC m=+1957.234873474" watchObservedRunningTime="2026-07-07 18:00:01.355657214 +0000 UTC m=+1957.237293743" Jul 07 18:00:01.371353 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:01.371284 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29724120-kpbdw" podStartSLOduration=1.371268428 podStartE2EDuration="1.371268428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:00:01.370243943 +0000 UTC m=+1957.251880470" watchObservedRunningTime="2026-07-07 18:00:01.371268428 +0000 UTC m=+1957.252904968" Jul 07 18:00:02.323206 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:02.323171 2562 generic.go:356] "Generic (PLEG): container finished" podID="151d029c-c12c-43f4-9d35-8304a02ab5c5" containerID="da938af4f64f965e70aad84703bc812b0ac6bfeba493cfcb73ca8eea667cfccd" exitCode=0 Jul 07 18:00:02.323634 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:02.323224 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j" event={"ID":"151d029c-c12c-43f4-9d35-8304a02ab5c5","Type":"ContainerDied","Data":"da938af4f64f965e70aad84703bc812b0ac6bfeba493cfcb73ca8eea667cfccd"} Jul 07 18:00:02.324782 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:02.324758 2562 generic.go:356] "Generic (PLEG): container finished" podID="3f43826b-2206-4855-8553-ca53cc16778c" containerID="26033f4332b15d55a93bb655121a7e2662b2926124d7ad1f756b03a924c4b29f" exitCode=0 Jul 07 18:00:02.324923 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:02.324824 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct" event={"ID":"3f43826b-2206-4855-8553-ca53cc16778c","Type":"ContainerDied","Data":"26033f4332b15d55a93bb655121a7e2662b2926124d7ad1f756b03a924c4b29f"} Jul 07 18:00:02.326288 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:02.326269 2562 generic.go:356] "Generic (PLEG): container finished" podID="33d6d987-77e9-4b1b-ba76-e3237eadcb15" containerID="69baf8f7852a7a6f3e5e191a376c77c02bb20d2687de330d39dab4887ab47fc0" exitCode=0 Jul 07 18:00:02.326379 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:02.326295 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724120-kpbdw" event={"ID":"33d6d987-77e9-4b1b-ba76-e3237eadcb15","Type":"ContainerDied","Data":"69baf8f7852a7a6f3e5e191a376c77c02bb20d2687de330d39dab4887ab47fc0"} Jul 07 18:00:03.478316 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:03.478278 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j" Jul 07 18:00:03.534219 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:03.534193 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct" Jul 07 18:00:03.537422 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:03.537398 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724120-kpbdw" Jul 07 18:00:03.573524 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:03.573496 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fm9wz\" (UniqueName: \"kubernetes.io/projected/151d029c-c12c-43f4-9d35-8304a02ab5c5-kube-api-access-fm9wz\") pod \"151d029c-c12c-43f4-9d35-8304a02ab5c5\" (UID: \"151d029c-c12c-43f4-9d35-8304a02ab5c5\") " Jul 07 18:00:03.575831 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:03.575798 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/151d029c-c12c-43f4-9d35-8304a02ab5c5-kube-api-access-fm9wz" (OuterVolumeSpecName: "kube-api-access-fm9wz") pod "151d029c-c12c-43f4-9d35-8304a02ab5c5" (UID: "151d029c-c12c-43f4-9d35-8304a02ab5c5"). InnerVolumeSpecName "kube-api-access-fm9wz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:00:03.674442 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:03.674344 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8m5vs\" (UniqueName: \"kubernetes.io/projected/33d6d987-77e9-4b1b-ba76-e3237eadcb15-kube-api-access-8m5vs\") pod \"33d6d987-77e9-4b1b-ba76-e3237eadcb15\" (UID: \"33d6d987-77e9-4b1b-ba76-e3237eadcb15\") " Jul 07 18:00:03.674442 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:03.674395 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5gz5\" (UniqueName: \"kubernetes.io/projected/3f43826b-2206-4855-8553-ca53cc16778c-kube-api-access-s5gz5\") pod \"3f43826b-2206-4855-8553-ca53cc16778c\" (UID: \"3f43826b-2206-4855-8553-ca53cc16778c\") " Jul 07 18:00:03.674680 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:03.674534 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fm9wz\" (UniqueName: \"kubernetes.io/projected/151d029c-c12c-43f4-9d35-8304a02ab5c5-kube-api-access-fm9wz\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:00:03.676767 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:03.676725 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33d6d987-77e9-4b1b-ba76-e3237eadcb15-kube-api-access-8m5vs" (OuterVolumeSpecName: "kube-api-access-8m5vs") pod "33d6d987-77e9-4b1b-ba76-e3237eadcb15" (UID: "33d6d987-77e9-4b1b-ba76-e3237eadcb15"). InnerVolumeSpecName "kube-api-access-8m5vs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:00:03.676767 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:03.676742 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f43826b-2206-4855-8553-ca53cc16778c-kube-api-access-s5gz5" (OuterVolumeSpecName: "kube-api-access-s5gz5") pod "3f43826b-2206-4855-8553-ca53cc16778c" (UID: "3f43826b-2206-4855-8553-ca53cc16778c"). InnerVolumeSpecName "kube-api-access-s5gz5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:00:03.775929 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:03.775891 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8m5vs\" (UniqueName: \"kubernetes.io/projected/33d6d987-77e9-4b1b-ba76-e3237eadcb15-kube-api-access-8m5vs\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:00:03.775929 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:03.775922 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5gz5\" (UniqueName: \"kubernetes.io/projected/3f43826b-2206-4855-8553-ca53cc16778c-kube-api-access-s5gz5\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:00:04.338719 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:04.338692 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j" Jul 07 18:00:04.338929 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:04.338739 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j" event={"ID":"151d029c-c12c-43f4-9d35-8304a02ab5c5","Type":"ContainerDied","Data":"8eb977cd8ea066abe7ddc3533d36f49e8472661bb0e5545881bc7d04aba2f742"} Jul 07 18:00:04.338929 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:04.338798 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8eb977cd8ea066abe7ddc3533d36f49e8472661bb0e5545881bc7d04aba2f742" Jul 07 18:00:04.340171 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:04.340146 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct" Jul 07 18:00:04.340312 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:04.340178 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct" event={"ID":"3f43826b-2206-4855-8553-ca53cc16778c","Type":"ContainerDied","Data":"6923714a93c48f4e00d295d397ef7c39a3129329c298e5214f0243059c40c321"} Jul 07 18:00:04.340312 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:04.340217 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6923714a93c48f4e00d295d397ef7c39a3129329c298e5214f0243059c40c321" Jul 07 18:00:04.341556 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:04.341523 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724120-kpbdw" event={"ID":"33d6d987-77e9-4b1b-ba76-e3237eadcb15","Type":"ContainerDied","Data":"a3cf6ca3dbdfe4319c3d90b2f9050d13e985bbba085adb7a4ffc23ad01f72ff5"} Jul 07 18:00:04.341556 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:04.341545 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724120-kpbdw" Jul 07 18:00:04.341556 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:04.341556 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a3cf6ca3dbdfe4319c3d90b2f9050d13e985bbba085adb7a4ffc23ad01f72ff5" Jul 07 18:00:11.690726 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:11.690680 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:00:12.641379 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:12.641342 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:00:12.808559 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:12.808526 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:00:13.234445 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:13.234406 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:00:13.334158 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:13.334118 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:00:13.438009 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:13.437956 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:00:13.576997 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:13.576936 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf"] Jul 07 18:00:13.577241 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:13.577216 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" podUID="cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54" containerName="maas-api" containerID="cri-o://bc34093fcf08b0e9343a0f4b4e979793e6e05c4b3652e0c7036a7de24d88e5e9" gracePeriod=30 Jul 07 18:00:13.605188 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:13.605157 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:00:13.610071 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:13.610027 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j"] Jul 07 18:00:13.613805 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:13.613786 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-c65a1b-29724120-jpg9j"] Jul 07 18:00:13.828056 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:13.827994 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" Jul 07 18:00:13.970743 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:13.970708 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8q6k\" (UniqueName: \"kubernetes.io/projected/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-kube-api-access-q8q6k\") pod \"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54\" (UID: \"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54\") " Jul 07 18:00:13.970904 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:13.970774 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-openshift-service-ca\") pod \"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54\" (UID: \"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54\") " Jul 07 18:00:13.970904 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:13.970858 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-maas-api-tls\") pod \"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54\" (UID: \"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54\") " Jul 07 18:00:13.971258 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:13.971236 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54" (UID: "cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:00:13.973057 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:13.973020 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54" (UID: "cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:00:13.973178 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:13.973129 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-kube-api-access-q8q6k" (OuterVolumeSpecName: "kube-api-access-q8q6k") pod "cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54" (UID: "cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54"). InnerVolumeSpecName "kube-api-access-q8q6k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:00:14.072493 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:14.072446 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-openshift-service-ca\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:00:14.072493 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:14.072488 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-maas-api-tls\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:00:14.072493 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:14.072497 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8q6k\" (UniqueName: \"kubernetes.io/projected/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54-kube-api-access-q8q6k\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:00:14.392879 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:14.392842 2562 generic.go:356] "Generic (PLEG): container finished" podID="cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54" containerID="bc34093fcf08b0e9343a0f4b4e979793e6e05c4b3652e0c7036a7de24d88e5e9" exitCode=0 Jul 07 18:00:14.393063 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:14.392893 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" Jul 07 18:00:14.393063 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:14.392923 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" event={"ID":"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54","Type":"ContainerDied","Data":"bc34093fcf08b0e9343a0f4b4e979793e6e05c4b3652e0c7036a7de24d88e5e9"} Jul 07 18:00:14.393063 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:14.393008 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf" event={"ID":"cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54","Type":"ContainerDied","Data":"aee106e74948a6713e806565665dd7686e8d5934bdb0929214ad37f235be2acc"} Jul 07 18:00:14.393063 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:14.393036 2562 scope.go:117] "RemoveContainer" containerID="bc34093fcf08b0e9343a0f4b4e979793e6e05c4b3652e0c7036a7de24d88e5e9" Jul 07 18:00:14.402438 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:14.402416 2562 scope.go:117] "RemoveContainer" containerID="bc34093fcf08b0e9343a0f4b4e979793e6e05c4b3652e0c7036a7de24d88e5e9" Jul 07 18:00:14.402759 ip-10-0-141-75 kubenswrapper[2562]: E0707 18:00:14.402741 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc34093fcf08b0e9343a0f4b4e979793e6e05c4b3652e0c7036a7de24d88e5e9\": container with ID starting with bc34093fcf08b0e9343a0f4b4e979793e6e05c4b3652e0c7036a7de24d88e5e9 not found: ID does not exist" containerID="bc34093fcf08b0e9343a0f4b4e979793e6e05c4b3652e0c7036a7de24d88e5e9" Jul 07 18:00:14.402809 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:14.402768 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc34093fcf08b0e9343a0f4b4e979793e6e05c4b3652e0c7036a7de24d88e5e9"} err="failed to get container status \"bc34093fcf08b0e9343a0f4b4e979793e6e05c4b3652e0c7036a7de24d88e5e9\": rpc error: code = NotFound desc = could not find container \"bc34093fcf08b0e9343a0f4b4e979793e6e05c4b3652e0c7036a7de24d88e5e9\": container with ID starting with bc34093fcf08b0e9343a0f4b4e979793e6e05c4b3652e0c7036a7de24d88e5e9 not found: ID does not exist" Jul 07 18:00:14.420439 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:14.420413 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf"] Jul 07 18:00:14.426464 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:14.426444 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-c65a1b-84b99f5cdb-w87lf"] Jul 07 18:00:14.637845 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:14.637810 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="151d029c-c12c-43f4-9d35-8304a02ab5c5" path="/var/lib/kubelet/pods/151d029c-c12c-43f4-9d35-8304a02ab5c5/volumes" Jul 07 18:00:14.638222 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:14.638208 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54" path="/var/lib/kubelet/pods/cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54/volumes" Jul 07 18:00:17.021698 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:17.021614 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:00:18.307209 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:18.307174 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:00:56.080002 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:56.079926 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:00:56.373293 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:56.373193 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:00:56.621189 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:56.621151 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:00:56.808594 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:56.808498 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:00:57.259207 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:57.259178 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:00:57.430863 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:57.430826 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:00:57.532933 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:57.532845 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:00:57.706269 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:57.706234 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz"] Jul 07 18:00:57.706538 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:57.706506 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" podUID="da56bd28-4170-49cf-997c-84817ff81a8b" containerName="maas-api" containerID="cri-o://d448990b891decd62be14bff86a82fecfeeeaae21f9dc898bda5d39893076b7d" gracePeriod=30 Jul 07 18:00:57.740327 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:57.740298 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:00:57.748815 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:57.748782 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct"] Jul 07 18:00:57.750981 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:57.750942 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-28127c-29724120-hgcct"] Jul 07 18:00:57.967697 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:57.967668 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" Jul 07 18:00:58.065083 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.065050 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da56bd28-4170-49cf-997c-84817ff81a8b-maas-api-tls\") pod \"da56bd28-4170-49cf-997c-84817ff81a8b\" (UID: \"da56bd28-4170-49cf-997c-84817ff81a8b\") " Jul 07 18:00:58.065259 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.065122 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da56bd28-4170-49cf-997c-84817ff81a8b-openshift-service-ca\") pod \"da56bd28-4170-49cf-997c-84817ff81a8b\" (UID: \"da56bd28-4170-49cf-997c-84817ff81a8b\") " Jul 07 18:00:58.065259 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.065192 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8wkx\" (UniqueName: \"kubernetes.io/projected/da56bd28-4170-49cf-997c-84817ff81a8b-kube-api-access-t8wkx\") pod \"da56bd28-4170-49cf-997c-84817ff81a8b\" (UID: \"da56bd28-4170-49cf-997c-84817ff81a8b\") " Jul 07 18:00:58.065635 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.065585 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da56bd28-4170-49cf-997c-84817ff81a8b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "da56bd28-4170-49cf-997c-84817ff81a8b" (UID: "da56bd28-4170-49cf-997c-84817ff81a8b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:00:58.067376 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.067349 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da56bd28-4170-49cf-997c-84817ff81a8b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "da56bd28-4170-49cf-997c-84817ff81a8b" (UID: "da56bd28-4170-49cf-997c-84817ff81a8b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:00:58.067716 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.067700 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da56bd28-4170-49cf-997c-84817ff81a8b-kube-api-access-t8wkx" (OuterVolumeSpecName: "kube-api-access-t8wkx") pod "da56bd28-4170-49cf-997c-84817ff81a8b" (UID: "da56bd28-4170-49cf-997c-84817ff81a8b"). InnerVolumeSpecName "kube-api-access-t8wkx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:00:58.165978 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.165923 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8wkx\" (UniqueName: \"kubernetes.io/projected/da56bd28-4170-49cf-997c-84817ff81a8b-kube-api-access-t8wkx\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:00:58.165978 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.165954 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da56bd28-4170-49cf-997c-84817ff81a8b-maas-api-tls\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:00:58.165978 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.165977 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da56bd28-4170-49cf-997c-84817ff81a8b-openshift-service-ca\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:00:58.626024 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.625980 2562 generic.go:356] "Generic (PLEG): container finished" podID="da56bd28-4170-49cf-997c-84817ff81a8b" containerID="d448990b891decd62be14bff86a82fecfeeeaae21f9dc898bda5d39893076b7d" exitCode=0 Jul 07 18:00:58.626024 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.626004 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" event={"ID":"da56bd28-4170-49cf-997c-84817ff81a8b","Type":"ContainerDied","Data":"d448990b891decd62be14bff86a82fecfeeeaae21f9dc898bda5d39893076b7d"} Jul 07 18:00:58.626478 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.626057 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" Jul 07 18:00:58.626478 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.626115 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz" event={"ID":"da56bd28-4170-49cf-997c-84817ff81a8b","Type":"ContainerDied","Data":"0dd75832cd5f99982eb3a6af4d2366fa8b5bb49acee4c42346b028f76fc699e5"} Jul 07 18:00:58.626478 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.626150 2562 scope.go:117] "RemoveContainer" containerID="d448990b891decd62be14bff86a82fecfeeeaae21f9dc898bda5d39893076b7d" Jul 07 18:00:58.636443 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.636423 2562 scope.go:117] "RemoveContainer" containerID="d448990b891decd62be14bff86a82fecfeeeaae21f9dc898bda5d39893076b7d" Jul 07 18:00:58.636767 ip-10-0-141-75 kubenswrapper[2562]: E0707 18:00:58.636742 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d448990b891decd62be14bff86a82fecfeeeaae21f9dc898bda5d39893076b7d\": container with ID starting with d448990b891decd62be14bff86a82fecfeeeaae21f9dc898bda5d39893076b7d not found: ID does not exist" containerID="d448990b891decd62be14bff86a82fecfeeeaae21f9dc898bda5d39893076b7d" Jul 07 18:00:58.636871 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.636776 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d448990b891decd62be14bff86a82fecfeeeaae21f9dc898bda5d39893076b7d"} err="failed to get container status \"d448990b891decd62be14bff86a82fecfeeeaae21f9dc898bda5d39893076b7d\": rpc error: code = NotFound desc = could not find container \"d448990b891decd62be14bff86a82fecfeeeaae21f9dc898bda5d39893076b7d\": container with ID starting with d448990b891decd62be14bff86a82fecfeeeaae21f9dc898bda5d39893076b7d not found: ID does not exist" Jul 07 18:00:58.639724 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.639700 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f43826b-2206-4855-8553-ca53cc16778c" path="/var/lib/kubelet/pods/3f43826b-2206-4855-8553-ca53cc16778c/volumes" Jul 07 18:00:58.651045 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.651016 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz"] Jul 07 18:00:58.656544 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:58.656518 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-28127c-f64c6df78-xflfz"] Jul 07 18:00:59.602722 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:00:59.602681 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:00.427627 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:00.427585 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:01:00.638239 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:00.638203 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da56bd28-4170-49cf-997c-84817ff81a8b" path="/var/lib/kubelet/pods/da56bd28-4170-49cf-997c-84817ff81a8b/volumes" Jul 07 18:01:03.657074 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:03.657034 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:03.720228 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:03.720189 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:03.966235 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:03.966126 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:04.209548 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:04.209513 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:04.602637 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:04.602597 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:04.858864 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:04.858774 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:04.942031 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:04.941993 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:05.094295 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:05.094258 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:07.741281 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.741248 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:07.831421 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.831386 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj"] Jul 07 18:01:07.831939 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.831919 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33d6d987-77e9-4b1b-ba76-e3237eadcb15" containerName="cleanup" Jul 07 18:01:07.831939 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.831941 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="33d6d987-77e9-4b1b-ba76-e3237eadcb15" containerName="cleanup" Jul 07 18:01:07.832143 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.831985 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="151d029c-c12c-43f4-9d35-8304a02ab5c5" containerName="cleanup" Jul 07 18:01:07.832143 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.831995 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="151d029c-c12c-43f4-9d35-8304a02ab5c5" containerName="cleanup" Jul 07 18:01:07.832143 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.832015 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f43826b-2206-4855-8553-ca53cc16778c" containerName="cleanup" Jul 07 18:01:07.832143 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.832026 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f43826b-2206-4855-8553-ca53cc16778c" containerName="cleanup" Jul 07 18:01:07.832143 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.832041 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54" containerName="maas-api" Jul 07 18:01:07.832143 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.832047 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54" containerName="maas-api" Jul 07 18:01:07.832143 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.832056 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da56bd28-4170-49cf-997c-84817ff81a8b" containerName="maas-api" Jul 07 18:01:07.832143 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.832065 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="da56bd28-4170-49cf-997c-84817ff81a8b" containerName="maas-api" Jul 07 18:01:07.832143 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.832144 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="33d6d987-77e9-4b1b-ba76-e3237eadcb15" containerName="cleanup" Jul 07 18:01:07.832588 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.832159 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfe0f9a9-a48f-4b7e-bfdf-ffe49dab7c54" containerName="maas-api" Jul 07 18:01:07.832588 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.832170 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="da56bd28-4170-49cf-997c-84817ff81a8b" containerName="maas-api" Jul 07 18:01:07.832588 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.832181 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f43826b-2206-4855-8553-ca53cc16778c" containerName="cleanup" Jul 07 18:01:07.832588 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.832194 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="151d029c-c12c-43f4-9d35-8304a02ab5c5" containerName="cleanup" Jul 07 18:01:07.832791 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.832757 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:07.836909 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.836890 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" Jul 07 18:01:07.839760 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.839737 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-1e4fca-serving-cert\"" Jul 07 18:01:07.839879 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.839843 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-q97km\"" Jul 07 18:01:07.846780 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.846759 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj"] Jul 07 18:01:07.908632 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.908604 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:07.967889 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.967852 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26qt2\" (UniqueName: \"kubernetes.io/projected/19dabc60-af46-4f19-a70c-930e8f1c3129-kube-api-access-26qt2\") pod \"maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj\" (UID: \"19dabc60-af46-4f19-a70c-930e8f1c3129\") " pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" Jul 07 18:01:07.968117 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.967897 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19dabc60-af46-4f19-a70c-930e8f1c3129-maas-api-tls\") pod \"maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj\" (UID: \"19dabc60-af46-4f19-a70c-930e8f1c3129\") " pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" Jul 07 18:01:07.968117 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:07.968085 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19dabc60-af46-4f19-a70c-930e8f1c3129-openshift-service-ca\") pod \"maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj\" (UID: \"19dabc60-af46-4f19-a70c-930e8f1c3129\") " pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" Jul 07 18:01:08.047516 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:08.047416 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:08.069615 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:08.069579 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26qt2\" (UniqueName: \"kubernetes.io/projected/19dabc60-af46-4f19-a70c-930e8f1c3129-kube-api-access-26qt2\") pod \"maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj\" (UID: \"19dabc60-af46-4f19-a70c-930e8f1c3129\") " pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" Jul 07 18:01:08.069775 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:08.069622 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19dabc60-af46-4f19-a70c-930e8f1c3129-maas-api-tls\") pod \"maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj\" (UID: \"19dabc60-af46-4f19-a70c-930e8f1c3129\") " pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" Jul 07 18:01:08.069775 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:08.069679 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19dabc60-af46-4f19-a70c-930e8f1c3129-openshift-service-ca\") pod \"maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj\" (UID: \"19dabc60-af46-4f19-a70c-930e8f1c3129\") " pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" Jul 07 18:01:08.070305 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:08.070280 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19dabc60-af46-4f19-a70c-930e8f1c3129-openshift-service-ca\") pod \"maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj\" (UID: \"19dabc60-af46-4f19-a70c-930e8f1c3129\") " pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" Jul 07 18:01:08.072279 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:08.072259 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19dabc60-af46-4f19-a70c-930e8f1c3129-maas-api-tls\") pod \"maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj\" (UID: \"19dabc60-af46-4f19-a70c-930e8f1c3129\") " pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" Jul 07 18:01:08.080399 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:08.080381 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26qt2\" (UniqueName: \"kubernetes.io/projected/19dabc60-af46-4f19-a70c-930e8f1c3129-kube-api-access-26qt2\") pod \"maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj\" (UID: \"19dabc60-af46-4f19-a70c-930e8f1c3129\") " pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" Jul 07 18:01:08.149415 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:08.149373 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" Jul 07 18:01:08.300038 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:08.300012 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj"] Jul 07 18:01:08.301839 ip-10-0-141-75 kubenswrapper[2562]: W0707 18:01:08.301808 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19dabc60_af46_4f19_a70c_930e8f1c3129.slice/crio-2e75ca874846754dd31c35fc9978a1ceb9630bc6f0bcf37731d3787c2409a8b9 WatchSource:0}: Error finding container 2e75ca874846754dd31c35fc9978a1ceb9630bc6f0bcf37731d3787c2409a8b9: Status 404 returned error can't find the container with id 2e75ca874846754dd31c35fc9978a1ceb9630bc6f0bcf37731d3787c2409a8b9 Jul 07 18:01:08.303226 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:08.303191 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 18:01:08.555937 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:08.555770 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:08.658532 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:08.658495 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:08.680266 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:08.680225 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" event={"ID":"19dabc60-af46-4f19-a70c-930e8f1c3129","Type":"ContainerStarted","Data":"2e75ca874846754dd31c35fc9978a1ceb9630bc6f0bcf37731d3787c2409a8b9"} Jul 07 18:01:09.686678 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:09.686642 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" event={"ID":"19dabc60-af46-4f19-a70c-930e8f1c3129","Type":"ContainerStarted","Data":"126ef429c8a1b5c6ef877166aa19801217fa1290d93cc4f6d89363108efaf958"} Jul 07 18:01:09.687106 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:09.686766 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" Jul 07 18:01:09.706754 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:09.706707 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" podStartSLOduration=2.175759447 podStartE2EDuration="2.706691618s" totalImagesPullingTime="530.932171ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:01:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:01:09.70555817 +0000 UTC m=+2025.587194746" watchObservedRunningTime="2026-07-07 18:01:09.706691618 +0000 UTC m=+2025.588328145" Jul 07 18:01:12.366278 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:12.366238 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:12.765879 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:12.765796 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:13.109726 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:13.109692 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:13.226510 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:13.226457 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:13.324859 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:13.324824 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:01:15.704735 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:15.704697 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" Jul 07 18:01:15.782933 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:15.782892 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:15.890086 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:15.890047 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:16.039831 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.039730 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r"] Jul 07 18:01:16.043443 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.043421 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" Jul 07 18:01:16.046375 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.046350 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-b93a8f-serving-cert\"" Jul 07 18:01:16.062707 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.062672 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r"] Jul 07 18:01:16.101855 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.101819 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:16.132716 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.132677 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0835f0eb-0151-49b5-9037-d4a9a315a436-maas-api-tls\") pod \"maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r\" (UID: \"0835f0eb-0151-49b5-9037-d4a9a315a436\") " pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" Jul 07 18:01:16.132890 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.132722 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6d78\" (UniqueName: \"kubernetes.io/projected/0835f0eb-0151-49b5-9037-d4a9a315a436-kube-api-access-n6d78\") pod \"maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r\" (UID: \"0835f0eb-0151-49b5-9037-d4a9a315a436\") " pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" Jul 07 18:01:16.132890 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.132786 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0835f0eb-0151-49b5-9037-d4a9a315a436-openshift-service-ca\") pod \"maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r\" (UID: \"0835f0eb-0151-49b5-9037-d4a9a315a436\") " pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" Jul 07 18:01:16.233423 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.233378 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0835f0eb-0151-49b5-9037-d4a9a315a436-openshift-service-ca\") pod \"maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r\" (UID: \"0835f0eb-0151-49b5-9037-d4a9a315a436\") " pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" Jul 07 18:01:16.233626 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.233494 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0835f0eb-0151-49b5-9037-d4a9a315a436-maas-api-tls\") pod \"maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r\" (UID: \"0835f0eb-0151-49b5-9037-d4a9a315a436\") " pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" Jul 07 18:01:16.233626 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.233532 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6d78\" (UniqueName: \"kubernetes.io/projected/0835f0eb-0151-49b5-9037-d4a9a315a436-kube-api-access-n6d78\") pod \"maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r\" (UID: \"0835f0eb-0151-49b5-9037-d4a9a315a436\") " pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" Jul 07 18:01:16.233749 ip-10-0-141-75 kubenswrapper[2562]: E0707 18:01:16.233661 2562 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-b93a8f-serving-cert: secret "maas-api-e2e-api-b-b93a8f-serving-cert" not found Jul 07 18:01:16.233801 ip-10-0-141-75 kubenswrapper[2562]: E0707 18:01:16.233750 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0835f0eb-0151-49b5-9037-d4a9a315a436-maas-api-tls podName:0835f0eb-0151-49b5-9037-d4a9a315a436 nodeName:}" failed. No retries permitted until 2026-07-07 18:01:16.733728754 +0000 UTC m=+2032.615365264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0835f0eb-0151-49b5-9037-d4a9a315a436-maas-api-tls") pod "maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" (UID: "0835f0eb-0151-49b5-9037-d4a9a315a436") : secret "maas-api-e2e-api-b-b93a8f-serving-cert" not found Jul 07 18:01:16.234106 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.234083 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0835f0eb-0151-49b5-9037-d4a9a315a436-openshift-service-ca\") pod \"maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r\" (UID: \"0835f0eb-0151-49b5-9037-d4a9a315a436\") " pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" Jul 07 18:01:16.245602 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.245570 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6d78\" (UniqueName: \"kubernetes.io/projected/0835f0eb-0151-49b5-9037-d4a9a315a436-kube-api-access-n6d78\") pod \"maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r\" (UID: \"0835f0eb-0151-49b5-9037-d4a9a315a436\") " pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" Jul 07 18:01:16.415790 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.415751 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:16.626377 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.626341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:16.737591 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.737490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0835f0eb-0151-49b5-9037-d4a9a315a436-maas-api-tls\") pod \"maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r\" (UID: \"0835f0eb-0151-49b5-9037-d4a9a315a436\") " pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" Jul 07 18:01:16.740090 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.740070 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0835f0eb-0151-49b5-9037-d4a9a315a436-maas-api-tls\") pod \"maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r\" (UID: \"0835f0eb-0151-49b5-9037-d4a9a315a436\") " pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" Jul 07 18:01:16.741386 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.741358 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:16.955614 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:16.955561 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" Jul 07 18:01:17.108342 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:17.108313 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r"] Jul 07 18:01:17.111343 ip-10-0-141-75 kubenswrapper[2562]: W0707 18:01:17.111315 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0835f0eb_0151_49b5_9037_d4a9a315a436.slice/crio-a0e9ba9d6a49202bf4216d545f0804058839a649a1ed9343e9e07c5c742bf1dd WatchSource:0}: Error finding container a0e9ba9d6a49202bf4216d545f0804058839a649a1ed9343e9e07c5c742bf1dd: Status 404 returned error can't find the container with id a0e9ba9d6a49202bf4216d545f0804058839a649a1ed9343e9e07c5c742bf1dd Jul 07 18:01:17.731715 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:17.731679 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" event={"ID":"0835f0eb-0151-49b5-9037-d4a9a315a436","Type":"ContainerStarted","Data":"68a56051678f8e1d3b814f9794788545c659e21440c3285439fdd82b8c4eec75"} Jul 07 18:01:17.731908 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:17.731740 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" event={"ID":"0835f0eb-0151-49b5-9037-d4a9a315a436","Type":"ContainerStarted","Data":"a0e9ba9d6a49202bf4216d545f0804058839a649a1ed9343e9e07c5c742bf1dd"} Jul 07 18:01:17.731908 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:17.731771 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" Jul 07 18:01:17.751781 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:17.751729 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" podStartSLOduration=1.305756635 podStartE2EDuration="1.751712879s" totalImagesPullingTime="445.956244ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:01:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:01:17.750586405 +0000 UTC m=+2033.632222931" watchObservedRunningTime="2026-07-07 18:01:17.751712879 +0000 UTC m=+2033.633349405" Jul 07 18:01:20.311426 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:20.311391 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:23.742890 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:23.742848 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" Jul 07 18:01:26.033424 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:26.033390 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:01:28.589541 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:28.589501 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:28.856577 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:28.856479 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:29.301154 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:29.301056 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:29.440542 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:29.440505 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:29.541473 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:29.541426 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:29.729982 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:29.729933 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj"] Jul 07 18:01:29.730434 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:29.730294 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" podUID="19dabc60-af46-4f19-a70c-930e8f1c3129" containerName="maas-api" containerID="cri-o://126ef429c8a1b5c6ef877166aa19801217fa1290d93cc4f6d89363108efaf958" gracePeriod=30 Jul 07 18:01:29.755685 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:29.755648 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:29.988825 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:29.988761 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" Jul 07 18:01:30.039908 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.039877 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19dabc60-af46-4f19-a70c-930e8f1c3129-openshift-service-ca\") pod \"19dabc60-af46-4f19-a70c-930e8f1c3129\" (UID: \"19dabc60-af46-4f19-a70c-930e8f1c3129\") " Jul 07 18:01:30.040248 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.040225 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19dabc60-af46-4f19-a70c-930e8f1c3129-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "19dabc60-af46-4f19-a70c-930e8f1c3129" (UID: "19dabc60-af46-4f19-a70c-930e8f1c3129"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:01:30.140379 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.140343 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19dabc60-af46-4f19-a70c-930e8f1c3129-maas-api-tls\") pod \"19dabc60-af46-4f19-a70c-930e8f1c3129\" (UID: \"19dabc60-af46-4f19-a70c-930e8f1c3129\") " Jul 07 18:01:30.140379 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.140381 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26qt2\" (UniqueName: \"kubernetes.io/projected/19dabc60-af46-4f19-a70c-930e8f1c3129-kube-api-access-26qt2\") pod \"19dabc60-af46-4f19-a70c-930e8f1c3129\" (UID: \"19dabc60-af46-4f19-a70c-930e8f1c3129\") " Jul 07 18:01:30.140581 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.140533 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19dabc60-af46-4f19-a70c-930e8f1c3129-openshift-service-ca\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:01:30.142675 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.142638 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19dabc60-af46-4f19-a70c-930e8f1c3129-kube-api-access-26qt2" (OuterVolumeSpecName: "kube-api-access-26qt2") pod "19dabc60-af46-4f19-a70c-930e8f1c3129" (UID: "19dabc60-af46-4f19-a70c-930e8f1c3129"). InnerVolumeSpecName "kube-api-access-26qt2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:01:30.142786 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.142748 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19dabc60-af46-4f19-a70c-930e8f1c3129-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "19dabc60-af46-4f19-a70c-930e8f1c3129" (UID: "19dabc60-af46-4f19-a70c-930e8f1c3129"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:01:30.241078 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.240994 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19dabc60-af46-4f19-a70c-930e8f1c3129-maas-api-tls\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:01:30.241078 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.241022 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26qt2\" (UniqueName: \"kubernetes.io/projected/19dabc60-af46-4f19-a70c-930e8f1c3129-kube-api-access-26qt2\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:01:30.800391 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.800352 2562 generic.go:356] "Generic (PLEG): container finished" podID="19dabc60-af46-4f19-a70c-930e8f1c3129" containerID="126ef429c8a1b5c6ef877166aa19801217fa1290d93cc4f6d89363108efaf958" exitCode=0 Jul 07 18:01:30.800797 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.800397 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" Jul 07 18:01:30.800797 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.800434 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" event={"ID":"19dabc60-af46-4f19-a70c-930e8f1c3129","Type":"ContainerDied","Data":"126ef429c8a1b5c6ef877166aa19801217fa1290d93cc4f6d89363108efaf958"} Jul 07 18:01:30.800797 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.800489 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj" event={"ID":"19dabc60-af46-4f19-a70c-930e8f1c3129","Type":"ContainerDied","Data":"2e75ca874846754dd31c35fc9978a1ceb9630bc6f0bcf37731d3787c2409a8b9"} Jul 07 18:01:30.800797 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.800510 2562 scope.go:117] "RemoveContainer" containerID="126ef429c8a1b5c6ef877166aa19801217fa1290d93cc4f6d89363108efaf958" Jul 07 18:01:30.809412 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.809391 2562 scope.go:117] "RemoveContainer" containerID="126ef429c8a1b5c6ef877166aa19801217fa1290d93cc4f6d89363108efaf958" Jul 07 18:01:30.809733 ip-10-0-141-75 kubenswrapper[2562]: E0707 18:01:30.809713 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"126ef429c8a1b5c6ef877166aa19801217fa1290d93cc4f6d89363108efaf958\": container with ID starting with 126ef429c8a1b5c6ef877166aa19801217fa1290d93cc4f6d89363108efaf958 not found: ID does not exist" containerID="126ef429c8a1b5c6ef877166aa19801217fa1290d93cc4f6d89363108efaf958" Jul 07 18:01:30.809780 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.809743 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"126ef429c8a1b5c6ef877166aa19801217fa1290d93cc4f6d89363108efaf958"} err="failed to get container status \"126ef429c8a1b5c6ef877166aa19801217fa1290d93cc4f6d89363108efaf958\": rpc error: code = NotFound desc = could not find container \"126ef429c8a1b5c6ef877166aa19801217fa1290d93cc4f6d89363108efaf958\": container with ID starting with 126ef429c8a1b5c6ef877166aa19801217fa1290d93cc4f6d89363108efaf958 not found: ID does not exist" Jul 07 18:01:30.821901 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.821865 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj"] Jul 07 18:01:30.826668 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:30.826638 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-1e4fca-9f68d6f69-h7jkj"] Jul 07 18:01:32.638032 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:32.638001 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19dabc60-af46-4f19-a70c-930e8f1c3129" path="/var/lib/kubelet/pods/19dabc60-af46-4f19-a70c-930e8f1c3129/volumes" Jul 07 18:01:35.791822 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:35.791785 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:35.857770 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:35.857733 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:36.132456 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:36.132421 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:36.402596 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:36.402504 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:36.557691 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:36.557649 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:36.957729 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:36.957683 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:37.112379 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.112346 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:37.203370 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.203327 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:37.396051 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.396010 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r"] Jul 07 18:01:37.396285 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.396261 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" podUID="0835f0eb-0151-49b5-9037-d4a9a315a436" containerName="maas-api" containerID="cri-o://68a56051678f8e1d3b814f9794788545c659e21440c3285439fdd82b8c4eec75" gracePeriod=30 Jul 07 18:01:37.435418 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.435386 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:37.664020 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.663928 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" Jul 07 18:01:37.699748 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.699712 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0835f0eb-0151-49b5-9037-d4a9a315a436-openshift-service-ca\") pod \"0835f0eb-0151-49b5-9037-d4a9a315a436\" (UID: \"0835f0eb-0151-49b5-9037-d4a9a315a436\") " Jul 07 18:01:37.699936 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.699760 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6d78\" (UniqueName: \"kubernetes.io/projected/0835f0eb-0151-49b5-9037-d4a9a315a436-kube-api-access-n6d78\") pod \"0835f0eb-0151-49b5-9037-d4a9a315a436\" (UID: \"0835f0eb-0151-49b5-9037-d4a9a315a436\") " Jul 07 18:01:37.699936 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.699780 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0835f0eb-0151-49b5-9037-d4a9a315a436-maas-api-tls\") pod \"0835f0eb-0151-49b5-9037-d4a9a315a436\" (UID: \"0835f0eb-0151-49b5-9037-d4a9a315a436\") " Jul 07 18:01:37.700210 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.700180 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0835f0eb-0151-49b5-9037-d4a9a315a436-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0835f0eb-0151-49b5-9037-d4a9a315a436" (UID: "0835f0eb-0151-49b5-9037-d4a9a315a436"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:01:37.702057 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.702030 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0835f0eb-0151-49b5-9037-d4a9a315a436-kube-api-access-n6d78" (OuterVolumeSpecName: "kube-api-access-n6d78") pod "0835f0eb-0151-49b5-9037-d4a9a315a436" (UID: "0835f0eb-0151-49b5-9037-d4a9a315a436"). InnerVolumeSpecName "kube-api-access-n6d78". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:01:37.702161 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.702053 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0835f0eb-0151-49b5-9037-d4a9a315a436-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0835f0eb-0151-49b5-9037-d4a9a315a436" (UID: "0835f0eb-0151-49b5-9037-d4a9a315a436"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:01:37.801063 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.801026 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0835f0eb-0151-49b5-9037-d4a9a315a436-openshift-service-ca\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:01:37.801063 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.801055 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6d78\" (UniqueName: \"kubernetes.io/projected/0835f0eb-0151-49b5-9037-d4a9a315a436-kube-api-access-n6d78\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:01:37.801063 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.801065 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0835f0eb-0151-49b5-9037-d4a9a315a436-maas-api-tls\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:01:37.840612 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.840577 2562 generic.go:356] "Generic (PLEG): container finished" podID="0835f0eb-0151-49b5-9037-d4a9a315a436" containerID="68a56051678f8e1d3b814f9794788545c659e21440c3285439fdd82b8c4eec75" exitCode=0 Jul 07 18:01:37.840802 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.840621 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" Jul 07 18:01:37.840802 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.840634 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" event={"ID":"0835f0eb-0151-49b5-9037-d4a9a315a436","Type":"ContainerDied","Data":"68a56051678f8e1d3b814f9794788545c659e21440c3285439fdd82b8c4eec75"} Jul 07 18:01:37.840802 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.840688 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r" event={"ID":"0835f0eb-0151-49b5-9037-d4a9a315a436","Type":"ContainerDied","Data":"a0e9ba9d6a49202bf4216d545f0804058839a649a1ed9343e9e07c5c742bf1dd"} Jul 07 18:01:37.840802 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.840718 2562 scope.go:117] "RemoveContainer" containerID="68a56051678f8e1d3b814f9794788545c659e21440c3285439fdd82b8c4eec75" Jul 07 18:01:37.849898 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.849877 2562 scope.go:117] "RemoveContainer" containerID="68a56051678f8e1d3b814f9794788545c659e21440c3285439fdd82b8c4eec75" Jul 07 18:01:37.850227 ip-10-0-141-75 kubenswrapper[2562]: E0707 18:01:37.850207 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68a56051678f8e1d3b814f9794788545c659e21440c3285439fdd82b8c4eec75\": container with ID starting with 68a56051678f8e1d3b814f9794788545c659e21440c3285439fdd82b8c4eec75 not found: ID does not exist" containerID="68a56051678f8e1d3b814f9794788545c659e21440c3285439fdd82b8c4eec75" Jul 07 18:01:37.850308 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.850235 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68a56051678f8e1d3b814f9794788545c659e21440c3285439fdd82b8c4eec75"} err="failed to get container status \"68a56051678f8e1d3b814f9794788545c659e21440c3285439fdd82b8c4eec75\": rpc error: code = NotFound desc = could not find container \"68a56051678f8e1d3b814f9794788545c659e21440c3285439fdd82b8c4eec75\": container with ID starting with 68a56051678f8e1d3b814f9794788545c659e21440c3285439fdd82b8c4eec75 not found: ID does not exist" Jul 07 18:01:37.865240 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.865213 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r"] Jul 07 18:01:37.871352 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:37.871326 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-b93a8f-64b754c68f-lx47r"] Jul 07 18:01:38.639391 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:38.639354 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0835f0eb-0151-49b5-9037-d4a9a315a436" path="/var/lib/kubelet/pods/0835f0eb-0151-49b5-9037-d4a9a315a436/volumes" Jul 07 18:01:39.919177 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:39.919143 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:01:43.653042 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:43.653006 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:43.753454 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:43.753418 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:43.814505 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:43.814468 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:44.014485 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.014369 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:44.387359 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.387326 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:44.694734 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.694641 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x"] Jul 07 18:01:44.695124 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.695053 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0835f0eb-0151-49b5-9037-d4a9a315a436" containerName="maas-api" Jul 07 18:01:44.695124 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.695066 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0835f0eb-0151-49b5-9037-d4a9a315a436" containerName="maas-api" Jul 07 18:01:44.695124 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.695076 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19dabc60-af46-4f19-a70c-930e8f1c3129" containerName="maas-api" Jul 07 18:01:44.695124 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.695081 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="19dabc60-af46-4f19-a70c-930e8f1c3129" containerName="maas-api" Jul 07 18:01:44.695257 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.695135 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="19dabc60-af46-4f19-a70c-930e8f1c3129" containerName="maas-api" Jul 07 18:01:44.695257 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.695143 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0835f0eb-0151-49b5-9037-d4a9a315a436" containerName="maas-api" Jul 07 18:01:44.699681 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.699660 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.703116 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.703094 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-8c0ae4-openshift-default-dockercfg-vwkvh\"" Jul 07 18:01:44.703265 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.703241 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 18:01:44.709664 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.709640 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x"] Jul 07 18:01:44.742475 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.742441 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:44.760725 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.760680 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-workload-socket\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.760924 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.760732 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-envoy\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.760924 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.760807 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-credential-socket\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.760924 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.760835 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-token\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.760924 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.760871 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istiod-ca-cert\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.760924 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.760924 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-workload-certs\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.761220 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.760940 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-data\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.761220 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.761019 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdl62\" (UniqueName: \"kubernetes.io/projected/a06c6658-09b3-4c00-a1b4-fa676d1a14df-kube-api-access-vdl62\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.761220 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.761042 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-podinfo\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.862228 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.862189 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-envoy\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.862228 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.862235 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-credential-socket\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.862507 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.862254 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-token\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.862507 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.862370 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istiod-ca-cert\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.862507 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.862440 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-workload-certs\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.862507 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.862476 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-data\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.862708 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.862534 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdl62\" (UniqueName: \"kubernetes.io/projected/a06c6658-09b3-4c00-a1b4-fa676d1a14df-kube-api-access-vdl62\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.862708 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.862575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-podinfo\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.862708 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.862616 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-workload-socket\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.862857 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.862838 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-workload-certs\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.862940 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.862917 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-workload-socket\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.863395 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.863366 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-data\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.863542 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.863516 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istiod-ca-cert\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.863678 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.863660 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-credential-socket\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.864919 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.864894 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-envoy\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.865372 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.865353 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-podinfo\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.872342 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.872311 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-token\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:44.872458 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:44.872320 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdl62\" (UniqueName: \"kubernetes.io/projected/a06c6658-09b3-4c00-a1b4-fa676d1a14df-kube-api-access-vdl62\") pod \"e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:45.010908 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:45.010807 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:45.176375 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:45.176302 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x"] Jul 07 18:01:45.185937 ip-10-0-141-75 kubenswrapper[2562]: W0707 18:01:45.185837 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda06c6658_09b3_4c00_a1b4_fa676d1a14df.slice/crio-27c000aed7ea6576e8a9efa95be8f7a7c3fde409ed0efe4114b8ac5c311e2dbe WatchSource:0}: Error finding container 27c000aed7ea6576e8a9efa95be8f7a7c3fde409ed0efe4114b8ac5c311e2dbe: Status 404 returned error can't find the container with id 27c000aed7ea6576e8a9efa95be8f7a7c3fde409ed0efe4114b8ac5c311e2dbe Jul 07 18:01:45.233462 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:45.233423 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:45.886026 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:45.885988 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" event={"ID":"a06c6658-09b3-4c00-a1b4-fa676d1a14df","Type":"ContainerStarted","Data":"27c000aed7ea6576e8a9efa95be8f7a7c3fde409ed0efe4114b8ac5c311e2dbe"} Jul 07 18:01:47.993368 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:47.993334 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 07 18:01:47.993672 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:47.993407 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 07 18:01:47.993672 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:47.993438 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 07 18:01:48.903540 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:48.903506 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" event={"ID":"a06c6658-09b3-4c00-a1b4-fa676d1a14df","Type":"ContainerStarted","Data":"94377f751609a3cecddffaf76fdc827548d7da7a3bc4a650fcd44bd7b2ad05b0"} Jul 07 18:01:48.929533 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:48.929476 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" podStartSLOduration=2.124071059 podStartE2EDuration="4.92945867s" totalImagesPullingTime="2.805387611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:01:48.927379957 +0000 UTC m=+2064.809016483" watchObservedRunningTime="2026-07-07 18:01:48.92945867 +0000 UTC m=+2064.811095193" Jul 07 18:01:49.011794 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:49.011757 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:49.016934 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:49.016904 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:49.368413 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:49.368382 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:49.908898 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:49.908850 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:49.909793 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:49.909772 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:01:50.889414 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:50.889371 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:50.970104 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:50.970047 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq"] Jul 07 18:01:50.970297 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:50.970166 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:50.974424 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:50.974403 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" Jul 07 18:01:50.978574 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:50.978554 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-q97km\"" Jul 07 18:01:50.978695 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:50.978591 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-8c0ae4-serving-cert\"" Jul 07 18:01:50.985690 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:50.985667 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq"] Jul 07 18:01:51.018152 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:51.018117 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq\" (UID: \"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1\") " pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" Jul 07 18:01:51.018357 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:51.018333 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lrjk\" (UniqueName: \"kubernetes.io/projected/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-kube-api-access-8lrjk\") pod \"maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq\" (UID: \"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1\") " pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" Jul 07 18:01:51.018422 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:51.018406 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-maas-api-tls\") pod \"maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq\" (UID: \"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1\") " pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" Jul 07 18:01:51.048036 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:51.047999 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:51.119831 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:51.119797 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq\" (UID: \"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1\") " pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" Jul 07 18:01:51.120048 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:51.119860 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lrjk\" (UniqueName: \"kubernetes.io/projected/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-kube-api-access-8lrjk\") pod \"maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq\" (UID: \"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1\") " pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" Jul 07 18:01:51.120048 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:51.119893 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-maas-api-tls\") pod \"maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq\" (UID: \"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1\") " pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" Jul 07 18:01:51.120550 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:51.120523 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq\" (UID: \"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1\") " pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" Jul 07 18:01:51.122522 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:51.122501 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-maas-api-tls\") pod \"maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq\" (UID: \"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1\") " pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" Jul 07 18:01:51.133519 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:51.133497 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lrjk\" (UniqueName: \"kubernetes.io/projected/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-kube-api-access-8lrjk\") pod \"maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq\" (UID: \"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1\") " pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" Jul 07 18:01:51.147634 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:51.147561 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:51.287156 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:51.287123 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" Jul 07 18:01:51.436919 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:51.436896 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq"] Jul 07 18:01:51.439464 ip-10-0-141-75 kubenswrapper[2562]: W0707 18:01:51.439433 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f7feda9_ef09_4220_a9cd_4d8eb92ca7e1.slice/crio-feee4e52e7f449c5fc50b33440dff5d55d868699de0844ebb3948ea3aa6f0a6c WatchSource:0}: Error finding container feee4e52e7f449c5fc50b33440dff5d55d868699de0844ebb3948ea3aa6f0a6c: Status 404 returned error can't find the container with id feee4e52e7f449c5fc50b33440dff5d55d868699de0844ebb3948ea3aa6f0a6c Jul 07 18:01:51.483758 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:51.483730 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:51.584553 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:51.584520 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:51.922845 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:51.922809 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" event={"ID":"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1","Type":"ContainerStarted","Data":"feee4e52e7f449c5fc50b33440dff5d55d868699de0844ebb3948ea3aa6f0a6c"} Jul 07 18:01:52.934137 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:52.934097 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" event={"ID":"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1","Type":"ContainerStarted","Data":"e744a59bd723f63bf695b47a6a1ee4f7a307ac2963def864f698dd5e4973698b"} Jul 07 18:01:52.934584 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:52.934202 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" Jul 07 18:01:52.955956 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:52.955904 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" podStartSLOduration=2.360490744 podStartE2EDuration="2.95588942s" totalImagesPullingTime="595.398676ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:01:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:01:52.954269812 +0000 UTC m=+2068.835906341" watchObservedRunningTime="2026-07-07 18:01:52.95588942 +0000 UTC m=+2068.837525951" Jul 07 18:01:54.228224 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:54.228187 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:01:55.438881 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:55.438842 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:55.799872 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:55.799771 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:56.168627 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:56.168578 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:56.308067 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:56.308033 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:58.945086 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:58.945053 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" Jul 07 18:01:59.026004 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:59.025946 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:59.128520 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:59.128486 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:59.273677 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:59.273569 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:59.508360 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:59.508310 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:59.869123 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:59.869058 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:01:59.964394 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:01:59.964357 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:02:01.994228 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:01.994196 2562 generic.go:356] "Generic (PLEG): container finished" podID="a06c6658-09b3-4c00-a1b4-fa676d1a14df" containerID="94377f751609a3cecddffaf76fdc827548d7da7a3bc4a650fcd44bd7b2ad05b0" exitCode=0 Jul 07 18:02:01.994722 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:01.994272 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" event={"ID":"a06c6658-09b3-4c00-a1b4-fa676d1a14df","Type":"ContainerDied","Data":"94377f751609a3cecddffaf76fdc827548d7da7a3bc4a650fcd44bd7b2ad05b0"} Jul 07 18:02:01.994722 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:01.994666 2562 scope.go:117] "RemoveContainer" containerID="94377f751609a3cecddffaf76fdc827548d7da7a3bc4a650fcd44bd7b2ad05b0" Jul 07 18:02:01.995646 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:01.995619 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 07 18:02:01.995737 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:01.995689 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 07 18:02:01.995737 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:01.995718 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 07 18:02:03.001795 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:03.001756 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" event={"ID":"a06c6658-09b3-4c00-a1b4-fa676d1a14df","Type":"ContainerStarted","Data":"43cc2944c102fdf72582c87201224307d0a634b5a49597f29f4f2f78ff5d7b65"} Jul 07 18:02:03.011013 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:03.010987 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:02:03.012298 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:03.012270 2562 patch_prober.go:28] interesting pod/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 07 18:02:03.012404 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:03.012324 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" podUID="a06c6658-09b3-4c00-a1b4-fa676d1a14df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 07 18:02:03.598466 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:03.598432 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:02:04.011709 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:04.011622 2562 patch_prober.go:28] interesting pod/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 07 18:02:04.012110 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:04.011708 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" podUID="a06c6658-09b3-4c00-a1b4-fa676d1a14df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 07 18:02:04.909192 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:04.909147 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:02:05.011541 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:05.011503 2562 patch_prober.go:28] interesting pod/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 07 18:02:05.011753 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:05.011566 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" podUID="a06c6658-09b3-4c00-a1b4-fa676d1a14df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 07 18:02:06.012134 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:06.012102 2562 patch_prober.go:28] interesting pod/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 07 18:02:06.012535 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:06.012157 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" podUID="a06c6658-09b3-4c00-a1b4-fa676d1a14df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 07 18:02:07.016132 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:07.016090 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:02:07.017004 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:07.016984 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:02:10.923341 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:10.923298 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:02:11.484267 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:11.484234 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:02:12.302186 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:12.302132 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:02:12.402782 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:12.402744 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:02:20.325291 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:20.325253 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:02:21.151256 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:21.151203 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:02:21.214941 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:21.214888 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:02:21.827034 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:21.826992 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:02:21.927502 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:21.927470 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:02:24.713738 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:24.713706 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 18:02:24.716064 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:02:24.716043 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 18:03:11.454631 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:03:11.454566 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:03:15.120512 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:03:15.120466 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:03:33.822205 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:03:33.822165 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:03:50.927145 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:03:50.927107 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:04:03.926421 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:04:03.926379 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:04:20.829118 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:04:20.829077 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:04:38.121881 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:04:38.121841 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:04:55.221914 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:04:55.221827 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:04:59.555539 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:04:59.555505 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:05:08.921489 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:05:08.921450 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:05:26.016642 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:05:26.016606 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:05:29.598475 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:05:29.598440 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:05:30.435009 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:05:30.434911 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:05:42.814869 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:05:42.814826 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:05:51.047530 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:05:51.047495 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:05:55.621604 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:05:55.621567 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:06:05.936060 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:06:05.935953 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:06:06.034211 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:06:06.034178 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:06:24.848150 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:06:24.848069 2562 scope.go:117] "RemoveContainer" containerID="da938af4f64f965e70aad84703bc812b0ac6bfeba493cfcb73ca8eea667cfccd" Jul 07 18:06:24.856940 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:06:24.856914 2562 scope.go:117] "RemoveContainer" containerID="26033f4332b15d55a93bb655121a7e2662b2926124d7ad1f756b03a924c4b29f" Jul 07 18:06:59.896323 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:06:59.896287 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:07:05.447019 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:07:05.446948 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:07:09.614446 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:07:09.614407 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:07:13.105354 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:07:13.105314 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:07:13.205789 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:07:13.205752 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:07:22.623382 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:07:22.623299 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:07:22.726584 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:07:22.726552 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:07:24.743932 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:07:24.743896 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 18:07:24.748087 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:07:24.748061 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 18:07:32.614357 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:07:32.614309 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:07:50.234900 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:07:50.234806 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:07:55.720087 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:07:55.720047 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:07:56.636924 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:07:56.636857 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:08:07.709532 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:08:07.709501 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:08:08.367903 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:08:08.367870 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:08:20.427091 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:08:20.427053 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:08:36.780614 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:08:36.780577 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:08:41.123314 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:08:41.123274 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:09:07.247916 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:07.247879 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:09:21.726498 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:21.726403 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:09:29.351617 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:29.351574 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:09:30.268369 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:30.268335 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:09:31.374761 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:31.374723 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:09:31.787249 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:31.787155 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:09:32.018799 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:32.018760 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:09:32.130381 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:32.130348 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:09:32.386579 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:32.386486 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq"] Jul 07 18:09:32.387049 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:32.386703 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" podUID="2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1" containerName="maas-api" containerID="cri-o://e744a59bd723f63bf695b47a6a1ee4f7a307ac2963def864f698dd5e4973698b" gracePeriod=30 Jul 07 18:09:32.424311 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:32.424270 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:09:32.645427 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:32.645359 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" Jul 07 18:09:32.657777 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:32.657748 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-openshift-service-ca\") pod \"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1\" (UID: \"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1\") " Jul 07 18:09:32.657915 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:32.657807 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lrjk\" (UniqueName: \"kubernetes.io/projected/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-kube-api-access-8lrjk\") pod \"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1\" (UID: \"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1\") " Jul 07 18:09:32.657915 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:32.657866 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-maas-api-tls\") pod \"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1\" (UID: \"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1\") " Jul 07 18:09:32.658228 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:32.658181 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1" (UID: "2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:09:32.660901 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:32.660853 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-kube-api-access-8lrjk" (OuterVolumeSpecName: "kube-api-access-8lrjk") pod "2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1" (UID: "2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1"). InnerVolumeSpecName "kube-api-access-8lrjk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:09:32.660901 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:32.660867 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1" (UID: "2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:09:32.758884 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:32.758834 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-maas-api-tls\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:09:32.758884 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:32.758882 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-openshift-service-ca\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:09:32.759145 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:32.758899 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lrjk\" (UniqueName: \"kubernetes.io/projected/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1-kube-api-access-8lrjk\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:09:33.346947 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:33.346908 2562 generic.go:356] "Generic (PLEG): container finished" podID="2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1" containerID="e744a59bd723f63bf695b47a6a1ee4f7a307ac2963def864f698dd5e4973698b" exitCode=0 Jul 07 18:09:33.347146 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:33.346957 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" Jul 07 18:09:33.347146 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:33.346992 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" event={"ID":"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1","Type":"ContainerDied","Data":"e744a59bd723f63bf695b47a6a1ee4f7a307ac2963def864f698dd5e4973698b"} Jul 07 18:09:33.347146 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:33.347055 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq" event={"ID":"2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1","Type":"ContainerDied","Data":"feee4e52e7f449c5fc50b33440dff5d55d868699de0844ebb3948ea3aa6f0a6c"} Jul 07 18:09:33.347146 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:33.347078 2562 scope.go:117] "RemoveContainer" containerID="e744a59bd723f63bf695b47a6a1ee4f7a307ac2963def864f698dd5e4973698b" Jul 07 18:09:33.357196 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:33.357176 2562 scope.go:117] "RemoveContainer" containerID="e744a59bd723f63bf695b47a6a1ee4f7a307ac2963def864f698dd5e4973698b" Jul 07 18:09:33.357508 ip-10-0-141-75 kubenswrapper[2562]: E0707 18:09:33.357481 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e744a59bd723f63bf695b47a6a1ee4f7a307ac2963def864f698dd5e4973698b\": container with ID starting with e744a59bd723f63bf695b47a6a1ee4f7a307ac2963def864f698dd5e4973698b not found: ID does not exist" containerID="e744a59bd723f63bf695b47a6a1ee4f7a307ac2963def864f698dd5e4973698b" Jul 07 18:09:33.357581 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:33.357520 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e744a59bd723f63bf695b47a6a1ee4f7a307ac2963def864f698dd5e4973698b"} err="failed to get container status \"e744a59bd723f63bf695b47a6a1ee4f7a307ac2963def864f698dd5e4973698b\": rpc error: code = NotFound desc = could not find container \"e744a59bd723f63bf695b47a6a1ee4f7a307ac2963def864f698dd5e4973698b\": container with ID starting with e744a59bd723f63bf695b47a6a1ee4f7a307ac2963def864f698dd5e4973698b not found: ID does not exist" Jul 07 18:09:33.372362 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:33.372336 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq"] Jul 07 18:09:33.377688 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:33.377663 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c0ae4-5fc54696df-k2ccq"] Jul 07 18:09:34.638261 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:34.638210 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1" path="/var/lib/kubelet/pods/2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1/volumes" Jul 07 18:09:36.856336 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:36.856306 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:09:41.938339 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:09:41.938304 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:10:14.340983 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:14.340936 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:10:14.602069 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:14.601953 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x"] Jul 07 18:10:14.602254 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:14.602230 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" podUID="a06c6658-09b3-4c00-a1b4-fa676d1a14df" containerName="istio-proxy" containerID="cri-o://43cc2944c102fdf72582c87201224307d0a634b5a49597f29f4f2f78ff5d7b65" gracePeriod=30 Jul 07 18:10:15.496021 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:15.495956 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:10:15.595797 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:15.595755 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:10:15.772123 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:15.772048 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:10:19.029653 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.029613 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mv272"] Jul 07 18:10:19.874985 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.874897 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:10:19.984112 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.984076 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdl62\" (UniqueName: \"kubernetes.io/projected/a06c6658-09b3-4c00-a1b4-fa676d1a14df-kube-api-access-vdl62\") pod \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " Jul 07 18:10:19.984299 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.984139 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-podinfo\") pod \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " Jul 07 18:10:19.984299 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.984162 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-token\") pod \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " Jul 07 18:10:19.984299 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.984193 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-workload-certs\") pod \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " Jul 07 18:10:19.984299 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.984224 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-workload-socket\") pod \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " Jul 07 18:10:19.984299 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.984258 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istiod-ca-cert\") pod \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " Jul 07 18:10:19.984552 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.984337 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-data\") pod \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " Jul 07 18:10:19.984552 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.984365 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-envoy\") pod \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " Jul 07 18:10:19.984552 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.984387 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-credential-socket\") pod \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\" (UID: \"a06c6658-09b3-4c00-a1b4-fa676d1a14df\") " Jul 07 18:10:19.984704 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.984560 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a06c6658-09b3-4c00-a1b4-fa676d1a14df" (UID: "a06c6658-09b3-4c00-a1b4-fa676d1a14df"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:10:19.984704 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.984576 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a06c6658-09b3-4c00-a1b4-fa676d1a14df" (UID: "a06c6658-09b3-4c00-a1b4-fa676d1a14df"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:10:19.984809 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.984692 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a06c6658-09b3-4c00-a1b4-fa676d1a14df" (UID: "a06c6658-09b3-4c00-a1b4-fa676d1a14df"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:10:19.984809 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.984709 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-workload-certs\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:10:19.984809 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.984729 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-workload-socket\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:10:19.984809 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.984787 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a06c6658-09b3-4c00-a1b4-fa676d1a14df" (UID: "a06c6658-09b3-4c00-a1b4-fa676d1a14df"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:10:19.985533 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.985500 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-data" (OuterVolumeSpecName: "istio-data") pod "a06c6658-09b3-4c00-a1b4-fa676d1a14df" (UID: "a06c6658-09b3-4c00-a1b4-fa676d1a14df"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:10:19.986578 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.986554 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-token" (OuterVolumeSpecName: "istio-token") pod "a06c6658-09b3-4c00-a1b4-fa676d1a14df" (UID: "a06c6658-09b3-4c00-a1b4-fa676d1a14df"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:10:19.986670 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.986577 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a06c6658-09b3-4c00-a1b4-fa676d1a14df-kube-api-access-vdl62" (OuterVolumeSpecName: "kube-api-access-vdl62") pod "a06c6658-09b3-4c00-a1b4-fa676d1a14df" (UID: "a06c6658-09b3-4c00-a1b4-fa676d1a14df"). InnerVolumeSpecName "kube-api-access-vdl62". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:10:19.986670 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.986630 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a06c6658-09b3-4c00-a1b4-fa676d1a14df" (UID: "a06c6658-09b3-4c00-a1b4-fa676d1a14df"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:10:19.986852 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:19.986834 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a06c6658-09b3-4c00-a1b4-fa676d1a14df" (UID: "a06c6658-09b3-4c00-a1b4-fa676d1a14df"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 18:10:20.085271 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.085235 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-data\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:10:20.085271 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.085263 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-envoy\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:10:20.085271 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.085273 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a06c6658-09b3-4c00-a1b4-fa676d1a14df-credential-socket\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:10:20.085271 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.085281 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdl62\" (UniqueName: \"kubernetes.io/projected/a06c6658-09b3-4c00-a1b4-fa676d1a14df-kube-api-access-vdl62\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:10:20.085748 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.085291 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-podinfo\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:10:20.085748 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.085299 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istio-token\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:10:20.085748 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.085307 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a06c6658-09b3-4c00-a1b4-fa676d1a14df-istiod-ca-cert\") on node \"ip-10-0-141-75.ec2.internal\" DevicePath \"\"" Jul 07 18:10:20.597237 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.597199 2562 generic.go:356] "Generic (PLEG): container finished" podID="a06c6658-09b3-4c00-a1b4-fa676d1a14df" containerID="43cc2944c102fdf72582c87201224307d0a634b5a49597f29f4f2f78ff5d7b65" exitCode=0 Jul 07 18:10:20.597418 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.597260 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" Jul 07 18:10:20.597418 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.597282 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" event={"ID":"a06c6658-09b3-4c00-a1b4-fa676d1a14df","Type":"ContainerDied","Data":"43cc2944c102fdf72582c87201224307d0a634b5a49597f29f4f2f78ff5d7b65"} Jul 07 18:10:20.597418 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.597340 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x" event={"ID":"a06c6658-09b3-4c00-a1b4-fa676d1a14df","Type":"ContainerDied","Data":"27c000aed7ea6576e8a9efa95be8f7a7c3fde409ed0efe4114b8ac5c311e2dbe"} Jul 07 18:10:20.597418 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.597363 2562 scope.go:117] "RemoveContainer" containerID="43cc2944c102fdf72582c87201224307d0a634b5a49597f29f4f2f78ff5d7b65" Jul 07 18:10:20.607188 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.607168 2562 scope.go:117] "RemoveContainer" containerID="94377f751609a3cecddffaf76fdc827548d7da7a3bc4a650fcd44bd7b2ad05b0" Jul 07 18:10:20.615476 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.615459 2562 scope.go:117] "RemoveContainer" containerID="43cc2944c102fdf72582c87201224307d0a634b5a49597f29f4f2f78ff5d7b65" Jul 07 18:10:20.615743 ip-10-0-141-75 kubenswrapper[2562]: E0707 18:10:20.615719 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43cc2944c102fdf72582c87201224307d0a634b5a49597f29f4f2f78ff5d7b65\": container with ID starting with 43cc2944c102fdf72582c87201224307d0a634b5a49597f29f4f2f78ff5d7b65 not found: ID does not exist" containerID="43cc2944c102fdf72582c87201224307d0a634b5a49597f29f4f2f78ff5d7b65" Jul 07 18:10:20.615787 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.615759 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43cc2944c102fdf72582c87201224307d0a634b5a49597f29f4f2f78ff5d7b65"} err="failed to get container status \"43cc2944c102fdf72582c87201224307d0a634b5a49597f29f4f2f78ff5d7b65\": rpc error: code = NotFound desc = could not find container \"43cc2944c102fdf72582c87201224307d0a634b5a49597f29f4f2f78ff5d7b65\": container with ID starting with 43cc2944c102fdf72582c87201224307d0a634b5a49597f29f4f2f78ff5d7b65 not found: ID does not exist" Jul 07 18:10:20.615787 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.615782 2562 scope.go:117] "RemoveContainer" containerID="94377f751609a3cecddffaf76fdc827548d7da7a3bc4a650fcd44bd7b2ad05b0" Jul 07 18:10:20.616046 ip-10-0-141-75 kubenswrapper[2562]: E0707 18:10:20.616027 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94377f751609a3cecddffaf76fdc827548d7da7a3bc4a650fcd44bd7b2ad05b0\": container with ID starting with 94377f751609a3cecddffaf76fdc827548d7da7a3bc4a650fcd44bd7b2ad05b0 not found: ID does not exist" containerID="94377f751609a3cecddffaf76fdc827548d7da7a3bc4a650fcd44bd7b2ad05b0" Jul 07 18:10:20.616095 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.616052 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94377f751609a3cecddffaf76fdc827548d7da7a3bc4a650fcd44bd7b2ad05b0"} err="failed to get container status \"94377f751609a3cecddffaf76fdc827548d7da7a3bc4a650fcd44bd7b2ad05b0\": rpc error: code = NotFound desc = could not find container \"94377f751609a3cecddffaf76fdc827548d7da7a3bc4a650fcd44bd7b2ad05b0\": container with ID starting with 94377f751609a3cecddffaf76fdc827548d7da7a3bc4a650fcd44bd7b2ad05b0 not found: ID does not exist" Jul 07 18:10:20.621776 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.621755 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x"] Jul 07 18:10:20.626763 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.626739 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-8c0ae4-openshift-default-b78769b7f-dws5x"] Jul 07 18:10:20.638228 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:20.638200 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a06c6658-09b3-4c00-a1b4-fa676d1a14df" path="/var/lib/kubelet/pods/a06c6658-09b3-4c00-a1b4-fa676d1a14df/volumes" Jul 07 18:10:21.873343 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:21.873315 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:10:21.939703 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:21.939666 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:10:22.165625 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:22.165530 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:10:22.407768 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:22.407728 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:10:27.740284 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:27.740237 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jxgfh_c97f57d4-0f45-4ddb-b9eb-8ee38f89dca1/manager/0.log" Jul 07 18:10:27.879577 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:27.879543 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-h7kbp_4294403d-0f91-4e28-974c-e7e725239890/manager/0.log" Jul 07 18:10:28.138883 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:28.138842 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724105-kvctv_081f9a1e-b69b-4ad2-864a-39a22c3110a0/cleanup/0.log" Jul 07 18:10:28.251519 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:28.251482 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724120-kpbdw_33d6d987-77e9-4b1b-ba76-e3237eadcb15/cleanup/0.log" Jul 07 18:10:28.490053 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:28.489958 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-z2fhk_7de9ab54-6d76-466f-99d9-d4dadaf41c15/server/0.log" Jul 07 18:10:28.617097 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:28.617067 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-8kp6k_df5a9647-84dc-4717-91e0-0a8cb94da704/manager/0.log" Jul 07 18:10:28.738101 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:28.738074 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-88cd6769f-5jms8_38bb6dde-3263-4924-b9e0-935aefe95a56/copy-manifests/0.log" Jul 07 18:10:28.745537 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:28.745453 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-88cd6769f-5jms8_38bb6dde-3263-4924-b9e0-935aefe95a56/manager/0.log" Jul 07 18:10:29.172733 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:29.172707 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-9p5gm_3d372ea4-0f57-4f06-847d-f58796d329d3/postgres/0.log" Jul 07 18:10:29.938164 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:29.938097 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:10:30.673780 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:30.673748 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-pfh22_cae879d3-65ad-4ef6-8991-2d755cd3e40a/manager/0.log" Jul 07 18:10:31.053190 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:31.053099 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-t96jf_e8d55d8b-1c52-421d-9120-332ce2f8f2f5/registry-server/0.log" Jul 07 18:10:31.192720 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:31.192666 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-wnwpr_d4144f66-b5ed-4c1d-94ec-212faa19b71e/manager/0.log" Jul 07 18:10:31.326491 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:31.326458 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-mv272_c9b353bd-ef2e-4ddb-a232-fb74e2c9b4f6/limitador/0.log" Jul 07 18:10:32.086670 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:32.086642 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bbcb8bd4c-hkrzg_6f3e8cac-099e-4860-8b63-6d27ab9477aa/kube-auth-proxy/0.log" Jul 07 18:10:32.688257 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:32.688207 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fbfb58cb7-68cb8_b6c8843e-708e-4e0d-a538-e35c93a4beec/router/0.log" Jul 07 18:10:33.044407 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:33.044326 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-7bxgq_0070ea1e-c6fd-49ee-b56b-828ab1b0cad2/main/0.log" Jul 07 18:10:38.152522 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.152487 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rs4cf/must-gather-xbzx9"] Jul 07 18:10:38.153039 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.152866 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1" containerName="maas-api" Jul 07 18:10:38.153039 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.152878 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1" containerName="maas-api" Jul 07 18:10:38.153039 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.152887 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a06c6658-09b3-4c00-a1b4-fa676d1a14df" containerName="istio-proxy" Jul 07 18:10:38.153039 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.152892 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a06c6658-09b3-4c00-a1b4-fa676d1a14df" containerName="istio-proxy" Jul 07 18:10:38.153039 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.152898 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a06c6658-09b3-4c00-a1b4-fa676d1a14df" containerName="istio-proxy" Jul 07 18:10:38.153039 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.152904 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a06c6658-09b3-4c00-a1b4-fa676d1a14df" containerName="istio-proxy" Jul 07 18:10:38.153039 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.152989 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a06c6658-09b3-4c00-a1b4-fa676d1a14df" containerName="istio-proxy" Jul 07 18:10:38.153039 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.153002 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f7feda9-ef09-4220-a9cd-4d8eb92ca7e1" containerName="maas-api" Jul 07 18:10:38.153039 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.153012 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a06c6658-09b3-4c00-a1b4-fa676d1a14df" containerName="istio-proxy" Jul 07 18:10:38.156151 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.156134 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rs4cf/must-gather-xbzx9" Jul 07 18:10:38.159108 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.159086 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rs4cf\"/\"default-dockercfg-9j7qd\"" Jul 07 18:10:38.159242 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.159088 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rs4cf\"/\"openshift-service-ca.crt\"" Jul 07 18:10:38.159677 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.159655 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:10:38.160337 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.160322 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rs4cf\"/\"kube-root-ca.crt\"" Jul 07 18:10:38.166573 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.166551 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rs4cf/must-gather-xbzx9"] Jul 07 18:10:38.244498 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.244463 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fd45819f-d6db-4898-9f2b-3673f30c7c1f-must-gather-output\") pod \"must-gather-xbzx9\" (UID: \"fd45819f-d6db-4898-9f2b-3673f30c7c1f\") " pod="openshift-must-gather-rs4cf/must-gather-xbzx9" Jul 07 18:10:38.244691 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.244579 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnvj2\" (UniqueName: \"kubernetes.io/projected/fd45819f-d6db-4898-9f2b-3673f30c7c1f-kube-api-access-jnvj2\") pod \"must-gather-xbzx9\" (UID: \"fd45819f-d6db-4898-9f2b-3673f30c7c1f\") " pod="openshift-must-gather-rs4cf/must-gather-xbzx9" Jul 07 18:10:38.260423 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.260396 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:10:38.345887 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.345852 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnvj2\" (UniqueName: \"kubernetes.io/projected/fd45819f-d6db-4898-9f2b-3673f30c7c1f-kube-api-access-jnvj2\") pod \"must-gather-xbzx9\" (UID: \"fd45819f-d6db-4898-9f2b-3673f30c7c1f\") " pod="openshift-must-gather-rs4cf/must-gather-xbzx9" Jul 07 18:10:38.346089 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.345921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fd45819f-d6db-4898-9f2b-3673f30c7c1f-must-gather-output\") pod \"must-gather-xbzx9\" (UID: \"fd45819f-d6db-4898-9f2b-3673f30c7c1f\") " pod="openshift-must-gather-rs4cf/must-gather-xbzx9" Jul 07 18:10:38.346260 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.346245 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fd45819f-d6db-4898-9f2b-3673f30c7c1f-must-gather-output\") pod \"must-gather-xbzx9\" (UID: \"fd45819f-d6db-4898-9f2b-3673f30c7c1f\") " pod="openshift-must-gather-rs4cf/must-gather-xbzx9" Jul 07 18:10:38.356555 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.356527 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnvj2\" (UniqueName: \"kubernetes.io/projected/fd45819f-d6db-4898-9f2b-3673f30c7c1f-kube-api-access-jnvj2\") pod \"must-gather-xbzx9\" (UID: \"fd45819f-d6db-4898-9f2b-3673f30c7c1f\") " pod="openshift-must-gather-rs4cf/must-gather-xbzx9" Jul 07 18:10:38.466468 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.466373 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rs4cf/must-gather-xbzx9" Jul 07 18:10:38.605098 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.605069 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rs4cf/must-gather-xbzx9"] Jul 07 18:10:38.610586 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.610558 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 18:10:38.695043 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:38.694998 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rs4cf/must-gather-xbzx9" event={"ID":"fd45819f-d6db-4898-9f2b-3673f30c7c1f","Type":"ContainerStarted","Data":"48c314ac6c642029f03b840f4c3275b188b8f237dff80f7bb8321fec51abdeee"} Jul 07 18:10:40.708082 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:40.708029 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rs4cf/must-gather-xbzx9" event={"ID":"fd45819f-d6db-4898-9f2b-3673f30c7c1f","Type":"ContainerStarted","Data":"26b07ba9da05f1614e6036d992a2b8e937b446000cc1b0f32aaa48bd5fc63a62"} Jul 07 18:10:40.708082 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:40.708087 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rs4cf/must-gather-xbzx9" event={"ID":"fd45819f-d6db-4898-9f2b-3673f30c7c1f","Type":"ContainerStarted","Data":"975238b3ab206c3b52a5834d35efd789281f8d89dc5211f347fa60d3bd867d93"} Jul 07 18:10:40.728041 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:40.727949 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rs4cf/must-gather-xbzx9" podStartSLOduration=1.327661083 podStartE2EDuration="2.727927153s" totalImagesPullingTime="1.40026607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:10:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:10:40.725328332 +0000 UTC m=+2596.606964861" watchObservedRunningTime="2026-07-07 18:10:40.727927153 +0000 UTC m=+2596.609563681" Jul 07 18:10:41.659631 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:41.659592 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5nf29_658cefc3-6811-4b18-b3a2-3af9ac357158/global-pull-secret-syncer/0.log" Jul 07 18:10:41.826020 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:41.825985 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" Jul 07 18:10:41.985843 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:41.985768 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-75.ec2.internal_5e37f1de7e940fc5cfbd6ca0135ad2aa/haproxy/0.log" Jul 07 18:10:46.634510 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:46.634471 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-pfh22_cae879d3-65ad-4ef6-8991-2d755cd3e40a/manager/0.log" Jul 07 18:10:46.781585 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:46.781552 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-t96jf_e8d55d8b-1c52-421d-9120-332ce2f8f2f5/registry-server/0.log" Jul 07 18:10:46.973304 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:46.973173 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-wnwpr_d4144f66-b5ed-4c1d-94ec-212faa19b71e/manager/0.log" Jul 07 18:10:47.002214 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:47.002187 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-mv272_c9b353bd-ef2e-4ddb-a232-fb74e2c9b4f6/limitador/0.log" Jul 07 18:10:49.108134 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:49.108097 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-52xr5_2308b91e-a103-451e-b06f-fc5bb0b50c97/cluster-monitoring-operator/0.log" Jul 07 18:10:49.460107 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:49.459999 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dncvg_68e5e3ca-2621-475d-97e2-b0a891b4abd9/node-exporter/0.log" Jul 07 18:10:49.484851 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:49.484823 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dncvg_68e5e3ca-2621-475d-97e2-b0a891b4abd9/kube-rbac-proxy/0.log" Jul 07 18:10:49.513708 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:49.513681 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dncvg_68e5e3ca-2621-475d-97e2-b0a891b4abd9/init-textfile/0.log" Jul 07 18:10:51.530848 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:51.530818 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-45d77_88acb68c-7353-4a80-a631-e7fe9713cfb4/networking-console-plugin/0.log" Jul 07 18:10:52.207033 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.206908 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/1.log" Jul 07 18:10:52.215034 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.214944 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nxsw4_8eea6af9-b0e3-4bf5-9ee8-aa45037566a3/console-operator/2.log" Jul 07 18:10:52.488612 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.488511 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5"] Jul 07 18:10:52.497034 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.497005 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.506249 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.506220 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5"] Jul 07 18:10:52.596231 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.596197 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054-lib-modules\") pod \"perf-node-gather-daemonset-8r5l5\" (UID: \"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054\") " pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.596231 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.596239 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054-sys\") pod \"perf-node-gather-daemonset-8r5l5\" (UID: \"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054\") " pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.596691 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.596334 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054-proc\") pod \"perf-node-gather-daemonset-8r5l5\" (UID: \"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054\") " pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.596691 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.596393 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054-podres\") pod \"perf-node-gather-daemonset-8r5l5\" (UID: \"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054\") " pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.596691 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.596444 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx8c9\" (UniqueName: \"kubernetes.io/projected/3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054-kube-api-access-zx8c9\") pod \"perf-node-gather-daemonset-8r5l5\" (UID: \"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054\") " pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.697612 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.697563 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054-proc\") pod \"perf-node-gather-daemonset-8r5l5\" (UID: \"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054\") " pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.697811 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.697630 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054-podres\") pod \"perf-node-gather-daemonset-8r5l5\" (UID: \"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054\") " pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.697811 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.697660 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx8c9\" (UniqueName: \"kubernetes.io/projected/3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054-kube-api-access-zx8c9\") pod \"perf-node-gather-daemonset-8r5l5\" (UID: \"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054\") " pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.697811 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.697679 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054-proc\") pod \"perf-node-gather-daemonset-8r5l5\" (UID: \"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054\") " pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.697811 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.697776 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054-lib-modules\") pod \"perf-node-gather-daemonset-8r5l5\" (UID: \"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054\") " pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.697811 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.697808 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054-sys\") pod \"perf-node-gather-daemonset-8r5l5\" (UID: \"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054\") " pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.698079 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.697820 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054-podres\") pod \"perf-node-gather-daemonset-8r5l5\" (UID: \"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054\") " pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.698079 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.697957 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054-sys\") pod \"perf-node-gather-daemonset-8r5l5\" (UID: \"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054\") " pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.698079 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.697998 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054-lib-modules\") pod \"perf-node-gather-daemonset-8r5l5\" (UID: \"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054\") " pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.708509 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.708477 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx8c9\" (UniqueName: \"kubernetes.io/projected/3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054-kube-api-access-zx8c9\") pod \"perf-node-gather-daemonset-8r5l5\" (UID: \"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054\") " pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.810782 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.810754 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:52.835896 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:52.835866 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-mpf9p_8a560068-6fa5-4afa-96d5-06792a244208/download-server/0.log" Jul 07 18:10:53.022279 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:53.022237 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5"] Jul 07 18:10:53.037941 ip-10-0-141-75 kubenswrapper[2562]: W0707 18:10:53.037911 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3f1bf3e7_b67a_4e07_b44b_66d4bd0bd054.slice/crio-53f6862b23e9313de7b001b780dcba68cd6ac172e5d742b841fb0f508ffdcb77 WatchSource:0}: Error finding container 53f6862b23e9313de7b001b780dcba68cd6ac172e5d742b841fb0f508ffdcb77: Status 404 returned error can't find the container with id 53f6862b23e9313de7b001b780dcba68cd6ac172e5d742b841fb0f508ffdcb77 Jul 07 18:10:53.565532 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:53.565504 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-ds5pp_b6d93f73-0d7b-44c8-bd96-b707875dd0af/volume-data-source-validator/0.log" Jul 07 18:10:53.798799 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:53.798759 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" event={"ID":"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054","Type":"ContainerStarted","Data":"4a6480e0a3882b7f9ee2be3e535384984e76c4a52a99f67b660278a7d0bd3020"} Jul 07 18:10:53.799346 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:53.798819 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" event={"ID":"3f1bf3e7-b67a-4e07-b44b-66d4bd0bd054","Type":"ContainerStarted","Data":"53f6862b23e9313de7b001b780dcba68cd6ac172e5d742b841fb0f508ffdcb77"} Jul 07 18:10:53.799346 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:53.799009 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:10:53.818238 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:53.818123 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" podStartSLOduration=1.818107127 podStartE2EDuration="1.818107127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:10:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:10:53.816792739 +0000 UTC m=+2609.698429265" watchObservedRunningTime="2026-07-07 18:10:53.818107127 +0000 UTC m=+2609.699743632" Jul 07 18:10:54.779004 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:54.778958 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n4h5f_9dcbb03e-a464-408b-a012-8fc6a386ef16/dns/0.log" Jul 07 18:10:54.808353 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:54.808317 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n4h5f_9dcbb03e-a464-408b-a012-8fc6a386ef16/kube-rbac-proxy/0.log" Jul 07 18:10:54.966355 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:54.966328 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hsg2d_dfec4ce5-3bf7-4600-b522-9daca21cf8c8/dns-node-resolver/0.log" Jul 07 18:10:55.575420 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:55.575388 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fx768_84115d28-35c1-4c55-9d01-c0f85ccfda09/node-ca/0.log" Jul 07 18:10:56.870977 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:56.870947 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bbcb8bd4c-hkrzg_6f3e8cac-099e-4860-8b63-6d27ab9477aa/kube-auth-proxy/0.log" Jul 07 18:10:57.123674 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:57.123595 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fbfb58cb7-68cb8_b6c8843e-708e-4e0d-a538-e35c93a4beec/router/0.log" Jul 07 18:10:57.904412 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:57.904384 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mwh52_74900825-b343-4085-aee1-d3e8c0bf2064/serve-healthcheck-canary/0.log" Jul 07 18:10:58.445336 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:58.445302 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-5gff8_15b2d389-16ed-4641-bffb-91298e7516c2/insights-operator/0.log" Jul 07 18:10:58.446765 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:58.446743 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-5gff8_15b2d389-16ed-4641-bffb-91298e7516c2/insights-operator/1.log" Jul 07 18:10:58.473138 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:58.473107 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-25z9v_19788e63-8c11-4c84-9998-c2e28b6c5908/kube-rbac-proxy/0.log" Jul 07 18:10:58.500916 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:58.500880 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-25z9v_19788e63-8c11-4c84-9998-c2e28b6c5908/exporter/0.log" Jul 07 18:10:58.535237 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:58.535207 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-25z9v_19788e63-8c11-4c84-9998-c2e28b6c5908/extractor/0.log" Jul 07 18:10:59.815155 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:10:59.815127 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rs4cf/perf-node-gather-daemonset-8r5l5" Jul 07 18:11:01.017559 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:01.017520 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jxgfh_c97f57d4-0f45-4ddb-b9eb-8ee38f89dca1/manager/0.log" Jul 07 18:11:01.211990 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:01.211768 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-h7kbp_4294403d-0f91-4e28-974c-e7e725239890/manager/0.log" Jul 07 18:11:01.280238 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:01.280149 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724105-kvctv_081f9a1e-b69b-4ad2-864a-39a22c3110a0/cleanup/0.log" Jul 07 18:11:01.311487 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:01.311450 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724120-kpbdw_33d6d987-77e9-4b1b-ba76-e3237eadcb15/cleanup/0.log" Jul 07 18:11:01.450425 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:01.450398 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-z2fhk_7de9ab54-6d76-466f-99d9-d4dadaf41c15/server/0.log" Jul 07 18:11:01.548315 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:01.548234 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-8kp6k_df5a9647-84dc-4717-91e0-0a8cb94da704/manager/0.log" Jul 07 18:11:01.583540 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:01.583502 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-88cd6769f-5jms8_38bb6dde-3263-4924-b9e0-935aefe95a56/manager/0.log" Jul 07 18:11:01.613674 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:01.613641 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-88cd6769f-5jms8_38bb6dde-3263-4924-b9e0-935aefe95a56/copy-manifests/0.log" Jul 07 18:11:01.832845 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:01.832814 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-9p5gm_3d372ea4-0f57-4f06-847d-f58796d329d3/postgres/0.log" Jul 07 18:11:02.743524 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:02.743493 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bbcb8bd4c-hkrzg_6f3e8cac-099e-4860-8b63-6d27ab9477aa/kube-auth-proxy/0.log" Jul 07 18:11:02.893560 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:02.893527 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fbfb58cb7-68cb8_b6c8843e-708e-4e0d-a538-e35c93a4beec/router/0.log" Jul 07 18:11:04.270207 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:04.270135 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bbcb8bd4c-hkrzg_6f3e8cac-099e-4860-8b63-6d27ab9477aa/kube-auth-proxy/0.log" Jul 07 18:11:04.460471 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:04.460432 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fbfb58cb7-68cb8_b6c8843e-708e-4e0d-a538-e35c93a4beec/router/0.log" Jul 07 18:11:06.586832 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:06.586798 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:11:06.687498 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:06.687461 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-77mtq_55f38dc4-f679-4b32-8f4e-6c3313461f0c/konnectivity-agent/0.log" limit=1024 Jul 07 18:11:08.718639 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:08.718598 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-cg5hg_25624a89-9856-4260-bccc-02578c46bc23/migrator/0.log" Jul 07 18:11:08.745292 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:08.745260 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-cg5hg_25624a89-9856-4260-bccc-02578c46bc23/graceful-termination/0.log" Jul 07 18:11:09.157261 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:09.157225 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-mr9z9_b398fe77-3362-4eaa-b811-95a3d049fcb5/kube-storage-version-migrator-operator/1.log" Jul 07 18:11:09.158115 ip-10-0-141-75 kubenswrapper[2562]: I0707 18:11:09.158067 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-mr9z9_b398fe77-3362-4eaa-b811-95a3d049fcb5/kube-storage-version-migrator-operator/0.log"