Jul 06 18:57:09.017413 ip-10-0-131-230 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 18:57:09.017425 ip-10-0-131-230 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 18:57:09.017435 ip-10-0-131-230 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 18:57:09.017751 ip-10-0-131-230 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 18:57:19.201050 ip-10-0-131-230 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 18:57:19.201069 ip-10-0-131-230 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e9ac855a9f73411088dbf2b65504e30b -- Jul 06 18:59:19.526792 ip-10-0-131-230 systemd[1]: Starting Kubernetes Kubelet... Jul 06 18:59:19.912641 ip-10-0-131-230 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 18:59:19.912641 ip-10-0-131-230 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 18:59:19.912641 ip-10-0-131-230 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 18:59:19.912641 ip-10-0-131-230 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 18:59:19.912641 ip-10-0-131-230 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 18:59:19.913544 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.913422 2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 18:59:19.918779 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918764 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:59:19.918779 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918779 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918782 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918785 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918803 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918806 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918809 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918813 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918815 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918817 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918820 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918824 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918828 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918830 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918833 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918835 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918838 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918840 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918843 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918846 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918849 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:59:19.918843 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918851 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918854 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918857 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918859 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918861 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918864 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918867 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918869 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918871 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918874 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918876 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918879 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918881 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918883 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918886 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918889 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918891 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918894 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918896 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918899 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:59:19.919284 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918901 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918904 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918906 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918908 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918911 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918913 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918916 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918918 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918920 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918923 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918925 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918927 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918930 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918932 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918935 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918937 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918940 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918942 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918945 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918948 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:59:19.919755 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918951 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918953 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918955 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918958 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918960 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918963 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918965 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918967 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918971 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918974 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918976 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918979 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918981 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918984 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918986 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918988 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918990 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918993 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918995 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.918998 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:59:19.920215 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919001 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919004 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919007 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919009 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919012 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919015 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919019 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919022 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919025 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919036 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919039 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919042 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919044 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919047 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919050 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919052 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919056 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919059 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919722 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:59:19.920677 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919728 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919731 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919734 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919737 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919740 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919742 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919745 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919747 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919749 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919752 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919754 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919757 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919760 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919763 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919765 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919768 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919771 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919773 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919775 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919778 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:59:19.921111 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919781 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919784 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919786 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919789 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919791 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919794 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919796 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919799 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919801 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919803 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919806 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919809 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919812 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919814 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919816 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919823 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919826 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919828 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919832 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919834 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:59:19.921621 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919837 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919839 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919842 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919844 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919846 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919849 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919851 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919853 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919856 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919858 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919860 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919863 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919866 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919868 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919870 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919873 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919875 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919878 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919880 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:59:19.922094 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919883 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919885 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919888 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919890 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919893 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919895 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919897 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919900 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919903 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919907 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919909 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919911 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919915 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919918 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919921 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919923 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919928 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919931 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919933 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:59:19.922544 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919936 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919939 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919942 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919944 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919946 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919949 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919951 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919953 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919956 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919958 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919961 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919963 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919966 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919968 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919970 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919972 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919975 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919977 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919979 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.919982 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:59:19.922972 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920381 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920387 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920390 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920393 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920396 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920398 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920401 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920404 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920406 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920409 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920411 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920414 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920416 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920419 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920421 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920424 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920426 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920428 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920445 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920447 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:59:19.923409 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920450 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920452 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920455 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920457 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920460 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920462 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920465 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920468 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920470 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920473 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920475 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920477 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920480 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920482 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920485 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920488 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920490 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920493 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920495 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920498 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:59:19.923860 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920503 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920505 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920507 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920510 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920512 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920515 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920517 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920519 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920522 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920524 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920526 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920529 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920531 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920535 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920538 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920541 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920544 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920547 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920549 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:59:19.924303 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920552 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920554 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920556 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920559 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920561 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920564 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920566 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920568 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920570 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920573 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920576 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920578 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920580 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920584 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920587 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920589 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920591 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920594 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920596 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920598 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:59:19.924742 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920601 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920603 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920606 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920608 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920610 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920613 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920618 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920622 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920626 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920630 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920633 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920636 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920638 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920641 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920644 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920646 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920649 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920651 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920654 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:59:19.925171 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.920656 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920726 2557 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920735 2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920745 2557 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920752 2557 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920756 2557 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920761 2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920765 2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920770 2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920773 2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920776 2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920779 2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920782 2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920785 2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920788 2557 flags.go:64] FLAG: --cgroup-root="" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920790 2557 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920793 2557 flags.go:64] FLAG: --client-ca-file="" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920796 2557 flags.go:64] FLAG: --cloud-provider="external" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920798 2557 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920802 2557 flags.go:64] FLAG: --cluster-domain="" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920805 2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920808 2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920811 2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920814 2557 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 18:59:19.925603 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920817 2557 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920820 2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920823 2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920825 2557 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920828 2557 flags.go:64] FLAG: --contention-profiling="false" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920831 2557 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920834 2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920837 2557 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920839 2557 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920843 2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920846 2557 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920848 2557 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920852 2557 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920855 2557 flags.go:64] FLAG: --enable-server="true" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920859 2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920864 2557 flags.go:64] FLAG: --event-burst="100" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920867 2557 flags.go:64] FLAG: --event-qps="50" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920869 2557 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920872 2557 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920875 2557 flags.go:64] FLAG: --eviction-hard="" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920879 2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920883 2557 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920886 2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920889 2557 flags.go:64] FLAG: --eviction-soft="" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920891 2557 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 18:59:19.926164 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920894 2557 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920897 2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920900 2557 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920903 2557 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920905 2557 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920908 2557 flags.go:64] FLAG: --feature-gates="" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920911 2557 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920914 2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920917 2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920920 2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920922 2557 flags.go:64] FLAG: --healthz-port="10248" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920925 2557 flags.go:64] FLAG: --help="false" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920928 2557 flags.go:64] FLAG: --hostname-override="ip-10-0-131-230.ec2.internal" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920931 2557 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920933 2557 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920936 2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920939 2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920942 2557 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920945 2557 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920948 2557 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920954 2557 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920958 2557 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920961 2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920963 2557 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 18:59:19.926793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920966 2557 flags.go:64] FLAG: --kube-reserved="" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920969 2557 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920971 2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920974 2557 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920978 2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920980 2557 flags.go:64] FLAG: --lock-file="" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920983 2557 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920986 2557 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920989 2557 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.920997 2557 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921000 2557 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921003 2557 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921006 2557 flags.go:64] FLAG: --logging-format="text" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921008 2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921011 2557 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921014 2557 flags.go:64] FLAG: --manifest-url="" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921017 2557 flags.go:64] FLAG: --manifest-url-header="" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921020 2557 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921023 2557 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921027 2557 flags.go:64] FLAG: --max-pods="110" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921030 2557 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921033 2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921036 2557 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921038 2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921041 2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 18:59:19.927338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921044 2557 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921046 2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921053 2557 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921056 2557 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921065 2557 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921068 2557 flags.go:64] FLAG: --pod-cidr="" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921071 2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921076 2557 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921079 2557 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921081 2557 flags.go:64] FLAG: --pods-per-core="0" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921084 2557 flags.go:64] FLAG: --port="10250" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921088 2557 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921091 2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e57999e2abfa6207" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921094 2557 flags.go:64] FLAG: --qos-reserved="" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921096 2557 flags.go:64] FLAG: --read-only-port="10255" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921099 2557 flags.go:64] FLAG: --register-node="true" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921101 2557 flags.go:64] FLAG: --register-with-taints="" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921105 2557 flags.go:64] FLAG: --registry-burst="10" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921107 2557 flags.go:64] FLAG: --registry-qps="5" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921110 2557 flags.go:64] FLAG: --reserved-cpus="" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921113 2557 flags.go:64] FLAG: --reserved-memory="" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921116 2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921119 2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921122 2557 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921125 2557 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 18:59:19.927952 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921127 2557 flags.go:64] FLAG: --runonce="false" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921130 2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921133 2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921136 2557 flags.go:64] FLAG: --seccomp-default="false" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921156 2557 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921159 2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921162 2557 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921165 2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921168 2557 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921170 2557 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921173 2557 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921176 2557 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921179 2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921182 2557 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921185 2557 flags.go:64] FLAG: --system-cgroups="" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921187 2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921193 2557 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921195 2557 flags.go:64] FLAG: --tls-cert-file="" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921199 2557 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921204 2557 flags.go:64] FLAG: --tls-min-version="" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921206 2557 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921209 2557 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921211 2557 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921214 2557 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921217 2557 flags.go:64] FLAG: --v="2" Jul 06 18:59:19.928531 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921220 2557 flags.go:64] FLAG: --version="false" Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921224 2557 flags.go:64] FLAG: --vmodule="" Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921227 2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.921230 2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921365 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921376 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921379 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921382 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921385 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921387 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921389 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921392 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921394 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921397 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921399 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921401 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921404 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921406 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921409 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921411 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:59:19.929086 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921420 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921424 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921427 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921444 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921447 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921451 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921455 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921458 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921461 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921464 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921467 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921470 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921472 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921475 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921478 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921480 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921483 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921485 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:59:19.929558 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921488 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921490 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921492 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921495 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921498 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921500 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921502 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921504 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921506 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921509 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921511 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921514 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921516 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921518 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921521 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921523 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921526 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921528 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921531 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:59:19.930147 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921534 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921537 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921539 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921541 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921543 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921546 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921548 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921550 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921553 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921555 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921557 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921560 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921563 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921565 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921567 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921570 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921572 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921574 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921577 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921579 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921581 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:59:19.930952 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921584 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921586 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921588 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921591 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921593 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921595 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921597 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921600 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921602 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921605 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921607 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921611 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921614 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921616 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921618 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921621 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921625 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921627 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921630 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:59:19.931576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921632 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:59:19.932018 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921635 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:59:19.932018 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921637 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:59:19.932018 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921640 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:59:19.932018 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921642 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:59:19.932018 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.921644 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:59:19.932018 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.922379 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 18:59:19.932018 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.929515 2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 18:59:19.932018 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.929530 2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 18:59:19.932018 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.929549 2557 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 18:59:19.932018 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.929558 2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 18:59:19.932018 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930706 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:59:19.932018 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930721 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:59:19.932018 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930727 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:59:19.932018 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930733 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:59:19.932018 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930737 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930741 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930746 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930750 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930754 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930763 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930768 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930772 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930776 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930780 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930785 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930790 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930794 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930798 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930803 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930806 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930809 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930811 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930817 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930819 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:59:19.932365 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930822 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930825 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930827 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930832 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930836 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930881 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930900 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.930905 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931053 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931059 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931062 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931064 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931067 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931069 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931072 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931074 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931077 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931079 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931081 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:59:19.932839 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931084 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931086 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931089 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931091 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931095 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931097 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931100 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931102 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931104 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931107 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931109 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931112 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931114 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931119 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931122 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931125 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931127 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931130 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931133 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931136 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:59:19.933264 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931139 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931141 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931144 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931146 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931149 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931151 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931153 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931156 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931158 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931161 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931163 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931165 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931168 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931190 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931193 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931196 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931200 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931203 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931206 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931209 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:59:19.933741 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931211 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:59:19.934238 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931213 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:59:19.934238 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931216 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:59:19.934238 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931218 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:59:19.934238 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931221 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:59:19.934238 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931223 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:59:19.934238 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931225 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:59:19.934238 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931228 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:59:19.934238 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931230 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:59:19.934238 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931234 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:59:19.934238 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931238 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:59:19.934238 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931241 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:59:19.934238 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931243 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:59:19.934238 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931246 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:59:19.934238 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931248 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:59:19.934238 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931251 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:59:19.934616 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.931256 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 18:59:19.934616 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931400 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:59:19.934616 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931405 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:59:19.934616 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931408 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:59:19.934616 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931411 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:59:19.934616 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931413 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:59:19.934616 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931416 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:59:19.934616 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931419 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:59:19.934616 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931421 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:59:19.934616 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931424 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:59:19.934616 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931426 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:59:19.934616 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931448 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:59:19.934616 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931453 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:59:19.934616 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931456 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:59:19.934616 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931458 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931461 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931463 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931468 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931470 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931473 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931475 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931477 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931480 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931482 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931485 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931487 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931490 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931493 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931496 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931498 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931501 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931503 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931506 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931508 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:59:19.934960 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931511 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931513 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931516 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931518 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931520 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931523 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931526 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931528 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931531 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931533 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931536 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931538 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931542 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931545 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931548 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931550 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931553 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931555 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931558 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931560 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:59:19.935426 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931563 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931566 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931570 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931572 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931575 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931578 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931581 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931584 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931586 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931589 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931592 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931595 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931597 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931600 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931602 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931605 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931607 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931609 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931612 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:59:19.935885 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931614 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931616 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931619 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931621 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931623 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931626 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931628 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931630 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931633 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931635 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931638 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931641 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931643 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931645 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931648 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931650 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931652 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931655 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931657 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931660 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:59:19.936319 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931662 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:59:19.936796 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931664 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:59:19.936796 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931667 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:59:19.936796 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931669 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:59:19.936796 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931671 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:59:19.936796 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931675 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:59:19.936796 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:19.931677 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:59:19.936796 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.931682 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 18:59:19.936796 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.931800 2557 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 18:59:19.936796 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.933906 2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 18:59:19.936796 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.935184 2557 server.go:1019] "Starting client certificate rotation" Jul 06 18:59:19.936796 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.935277 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 18:59:19.936796 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.936012 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 18:59:19.937830 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.937814 2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 18:59:19.942895 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.942872 2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 18:59:19.951079 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.951062 2557 log.go:25] "Validated CRI v1 runtime API" Jul 06 18:59:19.953906 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.953890 2557 log.go:25] "Validated CRI v1 image API" Jul 06 18:59:19.954640 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.954628 2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 18:59:19.957188 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.957160 2557 fs.go:135] Filesystem UUIDs: map[34ccf4ee-29cd-41fd-b0d9-34cf13a184e1:/dev/nvme0n1p4 77fbbcee-cfbc-496b-91c9-d91d0848699b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 06 18:59:19.957262 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.957186 2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 18:59:19.962797 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.962694 2557 manager.go:216] Machine: {Timestamp:2026-07-06 18:59:19.961398099 +0000 UTC m=+0.322561880 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099858 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f317e4e464dfcf00b9ea2e4b0f87e SystemUUID:ec2f317e-4e46-4dfc-f00b-9ea2e4b0f87e BootID:e9ac855a-9f73-4110-88db-f2b65504e30b Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:69:18:4c:52:71 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:69:18:4c:52:71 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c6:57:0d:89:aa:3d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 18:59:19.962797 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.962789 2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 18:59:19.962910 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.962867 2557 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 18:59:19.965844 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.965819 2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 18:59:19.965993 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.965847 2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-230.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 18:59:19.966727 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.966716 2557 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 18:59:19.966756 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.966728 2557 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 18:59:19.966756 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.966736 2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 18:59:19.967454 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.967427 2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 18:59:19.969194 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.969184 2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 18:59:19.972357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.972345 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 06 18:59:19.972536 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.972527 2557 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 18:59:19.973001 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.972991 2557 kubelet.go:486] "Attempting to sync node with API server" Jul 06 18:59:19.973029 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.973011 2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 18:59:19.973029 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.973021 2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 18:59:19.973095 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.973034 2557 kubelet.go:398] "Adding apiserver pod source" Jul 06 18:59:19.973095 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.973046 2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 18:59:19.973472 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.973458 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 18:59:19.973555 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.973479 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 18:59:19.976883 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.976867 2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 18:59:19.979169 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.979155 2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 18:59:19.979225 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.979185 2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 18:59:19.981373 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.981357 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 18:59:19.981408 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.981380 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 18:59:19.981408 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.981386 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 18:59:19.981408 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.981391 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 18:59:19.981408 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.981400 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 18:59:19.981408 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.981405 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 18:59:19.981542 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.981413 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 18:59:19.981542 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.981418 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 18:59:19.981542 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.981425 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 18:59:19.981542 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.981445 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 18:59:19.981542 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.981454 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 18:59:19.981542 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.981459 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 18:59:19.982107 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.982098 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 18:59:19.982138 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.982107 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 18:59:19.985181 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.985170 2557 server.go:1268] "Started kubelet" Jul 06 18:59:19.985315 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.985262 2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 18:59:19.985315 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.985285 2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 18:59:19.985472 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.985336 2557 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 18:59:19.986029 ip-10-0-131-230 systemd[1]: Started Kubernetes Kubelet. Jul 06 18:59:19.987422 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.987399 2557 server.go:310] "Adding debug handlers to kubelet server" Jul 06 18:59:19.987609 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.987587 2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 18:59:19.990813 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:19.990793 2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 18:59:19.990910 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.990850 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 18:59:19.992573 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.992542 2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-230.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 18:59:19.992674 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:19.992657 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 18:59:19.992722 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:19.992666 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 18:59:19.993027 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:19.992164 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-230.ec2.internal.18bfc86fce0192ba default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-230.ec2.internal,UID:ip-10-0-131-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-230.ec2.internal,},FirstTimestamp:2026-07-06 18:59:19.98514041 +0000 UTC m=+0.346304193,LastTimestamp:2026-07-06 18:59:19.98514041 +0000 UTC m=+0.346304193,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-230.ec2.internal,}" Jul 06 18:59:19.994898 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.994885 2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 18:59:19.994939 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.994918 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 18:59:19.995089 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:19.995070 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-230.ec2.internal\" not found" Jul 06 18:59:19.995678 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.995607 2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 18:59:19.995797 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.995783 2557 factory.go:55] Registering systemd factory Jul 06 18:59:19.995961 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.995798 2557 factory.go:223] Registration of the systemd container factory successfully Jul 06 18:59:19.996095 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.995995 2557 factory.go:153] Registering CRI-O factory Jul 06 18:59:19.996095 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.996007 2557 factory.go:223] Registration of the crio container factory successfully Jul 06 18:59:19.996095 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.996058 2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 18:59:19.996095 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.996079 2557 factory.go:103] Registering Raw factory Jul 06 18:59:19.996095 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.996093 2557 manager.go:1196] Started watching for new ooms in manager Jul 06 18:59:19.996292 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.996125 2557 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 18:59:19.996292 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.996141 2557 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 18:59:19.996918 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.996897 2557 reconstruct.go:97] "Volume reconstruction finished" Jul 06 18:59:19.996918 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.996913 2557 reconciler.go:29] "Reconciler: start to sync state" Jul 06 18:59:19.997216 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:19.997203 2557 manager.go:349] Starting recovery of all containers Jul 06 18:59:19.998507 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:19.998478 2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 18:59:19.998607 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:19.998560 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 18:59:20.009338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.009312 2557 manager.go:354] Recovery completed Jul 06 18:59:20.013254 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.013238 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:59:20.015057 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.015035 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:59:20.015125 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.015069 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:59:20.015125 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.015080 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:59:20.015632 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.015617 2557 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 18:59:20.015632 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.015630 2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 18:59:20.015723 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.015649 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 06 18:59:20.015723 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.015631 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:59:20.016288 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.016274 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:59:20.016350 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.016302 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:59:20.016350 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.016314 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:59:20.016849 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.016837 2557 policy_none.go:49] "None policy: Start" Jul 06 18:59:20.016891 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.016851 2557 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 18:59:20.016891 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.016862 2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 18:59:20.017982 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.017970 2557 policy_none.go:47] "Start" Jul 06 18:59:20.031164 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.031094 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-230.ec2.internal.18bfc86fcfca0ed8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-230.ec2.internal,UID:ip-10-0-131-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-230.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-230.ec2.internal,},FirstTimestamp:2026-07-06 18:59:20.0150566 +0000 UTC m=+0.376220381,LastTimestamp:2026-07-06 18:59:20.0150566 +0000 UTC m=+0.376220381,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-230.ec2.internal,}" Jul 06 18:59:20.038391 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.038311 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-230.ec2.internal.18bfc86fcfca53e8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-230.ec2.internal,UID:ip-10-0-131-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-230.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-230.ec2.internal,},FirstTimestamp:2026-07-06 18:59:20.01507428 +0000 UTC m=+0.376238061,LastTimestamp:2026-07-06 18:59:20.01507428 +0000 UTC m=+0.376238061,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-230.ec2.internal,}" Jul 06 18:59:20.042679 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.042607 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-230.ec2.internal.18bfc86fcfca785b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-230.ec2.internal,UID:ip-10-0-131-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-230.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-230.ec2.internal,},FirstTimestamp:2026-07-06 18:59:20.015083611 +0000 UTC m=+0.376247393,LastTimestamp:2026-07-06 18:59:20.015083611 +0000 UTC m=+0.376247393,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-230.ec2.internal,}" Jul 06 18:59:20.052887 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.052826 2557 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-230.ec2.internal.18bfc86fcfca0ed8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-230.ec2.internal.18bfc86fcfca0ed8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-230.ec2.internal,UID:ip-10-0-131-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-230.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-230.ec2.internal,},FirstTimestamp:2026-07-06 18:59:20.0150566 +0000 UTC m=+0.376220381,LastTimestamp:2026-07-06 18:59:20.01628787 +0000 UTC m=+0.377451651,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-230.ec2.internal,}" Jul 06 18:59:20.059003 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.058990 2557 manager.go:337] "Starting Device Plugin manager" Jul 06 18:59:20.059089 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.059028 2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 18:59:20.059089 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.059038 2557 server.go:87] "Starting device plugin registration server" Jul 06 18:59:20.059306 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.059295 2557 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 18:59:20.059340 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.059309 2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 18:59:20.059425 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.059397 2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 18:59:20.059546 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.059512 2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 18:59:20.059546 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.059522 2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 18:59:20.060476 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.060460 2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 18:59:20.060541 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.060498 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-230.ec2.internal\" not found" Jul 06 18:59:20.061999 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.061934 2557 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-230.ec2.internal.18bfc86fcfca53e8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-230.ec2.internal.18bfc86fcfca53e8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-230.ec2.internal,UID:ip-10-0-131-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-230.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-230.ec2.internal,},FirstTimestamp:2026-07-06 18:59:20.01507428 +0000 UTC m=+0.376238061,LastTimestamp:2026-07-06 18:59:20.016308757 +0000 UTC m=+0.377472537,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-230.ec2.internal,}" Jul 06 18:59:20.071870 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.071815 2557 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-230.ec2.internal.18bfc86fcfca785b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-230.ec2.internal.18bfc86fcfca785b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-230.ec2.internal,UID:ip-10-0-131-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-230.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-230.ec2.internal,},FirstTimestamp:2026-07-06 18:59:20.015083611 +0000 UTC m=+0.376247393,LastTimestamp:2026-07-06 18:59:20.016318176 +0000 UTC m=+0.377481956,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-230.ec2.internal,}" Jul 06 18:59:20.083492 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.083405 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-230.ec2.internal.18bfc86fd28c4b5a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-230.ec2.internal,UID:ip-10-0-131-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-131-230.ec2.internal,},FirstTimestamp:2026-07-06 18:59:20.061340506 +0000 UTC m=+0.422504276,LastTimestamp:2026-07-06 18:59:20.061340506 +0000 UTC m=+0.422504276,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-230.ec2.internal,}" Jul 06 18:59:20.099608 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.099586 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p9mz5" Jul 06 18:59:20.113950 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.109853 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p9mz5" Jul 06 18:59:20.160296 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.160220 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:59:20.161158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.161132 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:59:20.161232 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.161177 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:59:20.161232 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.161187 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:59:20.161232 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.161213 2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.179689 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.179667 2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.179785 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.179693 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-230.ec2.internal\": node \"ip-10-0-131-230.ec2.internal\" not found" Jul 06 18:59:20.188737 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.188703 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 18:59:20.189981 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.189966 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 18:59:20.190054 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.190000 2557 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 18:59:20.190054 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.190021 2557 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 18:59:20.190149 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.190050 2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 18:59:20.195018 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.195001 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:59:20.230011 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.229986 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-230.ec2.internal\" not found" Jul 06 18:59:20.290335 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.290296 2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-230.ec2.internal"] Jul 06 18:59:20.290534 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.290373 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:59:20.291325 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.291309 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:59:20.291398 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.291338 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:59:20.291398 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.291351 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:59:20.292601 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.292589 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:59:20.292707 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.292693 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.292821 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.292724 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:59:20.293261 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.293245 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:59:20.293335 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.293272 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:59:20.293335 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.293282 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:59:20.293983 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.293967 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:59:20.294066 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.293994 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:59:20.294066 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.294005 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:59:20.295203 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.295189 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.295259 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.295220 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:59:20.297958 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.297942 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/924289aee6847fb393cec64c82071a0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal\" (UID: \"924289aee6847fb393cec64c82071a0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.298017 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.297965 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/924289aee6847fb393cec64c82071a0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal\" (UID: \"924289aee6847fb393cec64c82071a0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.298017 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.297981 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f78ec814e87d18142aa89255a77769d0-config\") pod \"kube-apiserver-proxy-ip-10-0-131-230.ec2.internal\" (UID: \"f78ec814e87d18142aa89255a77769d0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.298634 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.298549 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:59:20.298634 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.298576 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:59:20.298634 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.298586 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:59:20.308944 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.308926 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-230.ec2.internal\" not found" node="ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.314680 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.314666 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-230.ec2.internal\" not found" node="ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.330958 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.330939 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-230.ec2.internal\" not found" Jul 06 18:59:20.398677 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.398649 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/924289aee6847fb393cec64c82071a0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal\" (UID: \"924289aee6847fb393cec64c82071a0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.398677 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.398678 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/924289aee6847fb393cec64c82071a0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal\" (UID: \"924289aee6847fb393cec64c82071a0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.398847 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.398695 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f78ec814e87d18142aa89255a77769d0-config\") pod \"kube-apiserver-proxy-ip-10-0-131-230.ec2.internal\" (UID: \"f78ec814e87d18142aa89255a77769d0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.398847 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.398721 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f78ec814e87d18142aa89255a77769d0-config\") pod \"kube-apiserver-proxy-ip-10-0-131-230.ec2.internal\" (UID: \"f78ec814e87d18142aa89255a77769d0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.398847 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.398739 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/924289aee6847fb393cec64c82071a0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal\" (UID: \"924289aee6847fb393cec64c82071a0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.398847 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.398769 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/924289aee6847fb393cec64c82071a0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal\" (UID: \"924289aee6847fb393cec64c82071a0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.431173 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.431097 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-230.ec2.internal\" not found" Jul 06 18:59:20.532112 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.532076 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-230.ec2.internal\" not found" Jul 06 18:59:20.612609 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.612578 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.618963 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.618942 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-230.ec2.internal" Jul 06 18:59:20.632906 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.632888 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-230.ec2.internal\" not found" Jul 06 18:59:20.733428 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.733346 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-230.ec2.internal\" not found" Jul 06 18:59:20.833833 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.833803 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-230.ec2.internal\" not found" Jul 06 18:59:20.934617 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.934590 2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 18:59:20.934617 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:20.934587 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-230.ec2.internal\" not found" Jul 06 18:59:20.935139 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.934726 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 18:59:20.996943 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:20.996912 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:59:21.012925 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.011417 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:59:21.057005 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.056976 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:59:21.095872 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.095834 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal" Jul 06 18:59:21.111259 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.111223 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 18:59:21.112949 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.112932 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-230.ec2.internal" Jul 06 18:59:21.115385 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.115357 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 18:54:20 +0000 UTC" deadline="2028-02-15 16:42:39.137059814 +0000 UTC" Jul 06 18:59:21.115472 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.115385 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14133h43m18.021677432s" Jul 06 18:59:21.125725 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.125707 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 18:59:21.144286 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:21.144243 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod924289aee6847fb393cec64c82071a0e.slice/crio-38b90dbf2e67453305f1ea218223fd3464da14f4fee404a439a70c857cc53915 WatchSource:0}: Error finding container 38b90dbf2e67453305f1ea218223fd3464da14f4fee404a439a70c857cc53915: Status 404 returned error can't find the container with id 38b90dbf2e67453305f1ea218223fd3464da14f4fee404a439a70c857cc53915 Jul 06 18:59:21.144794 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:21.144774 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf78ec814e87d18142aa89255a77769d0.slice/crio-0e0af469d41367dc24f9a3519c7969e3b2814cf824eff9c5895d3e592fa00a33 WatchSource:0}: Error finding container 0e0af469d41367dc24f9a3519c7969e3b2814cf824eff9c5895d3e592fa00a33: Status 404 returned error can't find the container with id 0e0af469d41367dc24f9a3519c7969e3b2814cf824eff9c5895d3e592fa00a33 Jul 06 18:59:21.148796 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.148773 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 18:59:21.193826 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.193765 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-230.ec2.internal" event={"ID":"f78ec814e87d18142aa89255a77769d0","Type":"ContainerStarted","Data":"0e0af469d41367dc24f9a3519c7969e3b2814cf824eff9c5895d3e592fa00a33"} Jul 06 18:59:21.194844 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.194822 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal" event={"ID":"924289aee6847fb393cec64c82071a0e","Type":"ContainerStarted","Data":"38b90dbf2e67453305f1ea218223fd3464da14f4fee404a439a70c857cc53915"} Jul 06 18:59:21.974898 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.974868 2557 apiserver.go:52] "Watching apiserver" Jul 06 18:59:21.982392 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.982365 2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 18:59:21.982809 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.982782 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-qxm9t","openshift-cluster-node-tuning-operator/tuned-pg99r","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal","openshift-multus/multus-additional-cni-plugins-vjmjt","openshift-multus/multus-lsdps","openshift-multus/network-metrics-daemon-2hf6t","openshift-ovn-kubernetes/ovnkube-node-z6w5n","kube-system/global-pull-secret-syncer-k9pnm","kube-system/konnectivity-agent-2lj55","kube-system/kube-apiserver-proxy-ip-10-0-131-230.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5","openshift-dns/node-resolver-n8wgx","openshift-image-registry/node-ca-xn6qm","openshift-network-diagnostics/network-check-target-69gb8"] Jul 06 18:59:21.984499 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.984477 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:21.985756 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.985737 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xn6qm" Jul 06 18:59:21.986899 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.986876 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:21.986996 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:21.986947 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:21.987977 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.987954 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 18:59:21.988073 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.988042 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 18:59:21.988651 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.988371 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:21.989638 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.989594 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 18:59:21.989915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.989898 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lsdps" Jul 06 18:59:21.989989 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.989920 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wz5kg\"" Jul 06 18:59:21.990119 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.990102 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 18:59:21.991313 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.991291 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 18:59:21.991422 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.991338 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:21.991422 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:21.991398 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:21.991609 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.991591 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 18:59:21.991662 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.991647 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 18:59:21.991712 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.991696 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 18:59:21.991712 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.991704 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 18:59:21.991809 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.991719 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 18:59:21.991809 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.991769 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 18:59:21.992125 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.992107 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 18:59:21.992184 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.992151 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-czhzt\"" Jul 06 18:59:21.992227 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.992191 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 18:59:21.992422 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.992405 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 18:59:21.992422 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.992417 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wxkcb\"" Jul 06 18:59:21.992830 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.992578 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2lj55" Jul 06 18:59:21.993552 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.993533 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qxkdf\"" Jul 06 18:59:21.993723 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.993707 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 18:59:21.994414 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.994396 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qxm9t" Jul 06 18:59:21.995784 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.995735 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 18:59:21.997355 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.997141 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 18:59:21.997355 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.997164 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:21.997559 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.997543 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 18:59:21.998185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.998162 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-spxrn\"" Jul 06 18:59:21.999166 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.997542 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9wfqv\"" Jul 06 18:59:21.999166 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.998887 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 18:59:21.999166 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.998927 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 18:59:21.999166 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:21.999094 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 18:59:22.000390 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.000364 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:22.000494 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.000459 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:22.001217 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.001193 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 18:59:22.001573 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.001559 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7kq2n\"" Jul 06 18:59:22.001973 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.001957 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 18:59:22.002419 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.002397 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.004661 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.004621 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n8wgx" Jul 06 18:59:22.004947 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.004928 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kc5w5\"" Jul 06 18:59:22.005280 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.005231 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 18:59:22.005368 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.005317 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 18:59:22.005633 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.005616 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 18:59:22.007204 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.007185 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h4dbz\"" Jul 06 18:59:22.007281 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.007261 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 18:59:22.007878 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.007860 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 18:59:22.008742 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.008722 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b9d924cf-a6f6-47d0-adae-989ebcab4a23-serviceca\") pod \"node-ca-xn6qm\" (UID: \"b9d924cf-a6f6-47d0-adae-989ebcab4a23\") " pod="openshift-image-registry/node-ca-xn6qm" Jul 06 18:59:22.008836 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.008756 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv88d\" (UniqueName: \"kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d\") pod \"network-check-target-69gb8\" (UID: \"794f47cc-dce2-43e8-b571-d431f4b6cec0\") " pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:22.008836 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.008782 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-cnibin\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.008836 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.008805 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-run-k8s-cni-cncf-io\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.008836 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.008827 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-kubernetes\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.009043 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.008866 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-env-overrides\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.009043 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.008899 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-cni-binary-copy\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.009043 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.008924 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-var-lib-cni-bin\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.009043 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.008947 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-systemd\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.009043 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.008970 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30b42dfe-0e9e-441c-a4d8-c5db4383b157-cni-binary-copy\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.009043 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009013 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs\") pod \"network-metrics-daemon-2hf6t\" (UID: \"cd62ad65-3e90-46f3-bdb6-67b9cb3bd158\") " pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:22.009296 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009059 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.009296 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009092 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-system-cni-dir\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.009296 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009116 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-multus-cni-dir\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.009296 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009143 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-sysctl-d\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.009296 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009184 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-sys\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.009296 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009218 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-var-lib-openvswitch\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.009296 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009254 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-node-log\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.009296 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009282 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-run-netns\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.009688 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009304 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c-konnectivity-ca\") pod \"konnectivity-agent-2lj55\" (UID: \"d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c\") " pod="kube-system/konnectivity-agent-2lj55" Jul 06 18:59:22.009688 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009344 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6rkm\" (UniqueName: \"kubernetes.io/projected/1527a401-f8a5-46e8-b5e9-bbb508d666be-kube-api-access-g6rkm\") pod \"iptables-alerter-qxm9t\" (UID: \"1527a401-f8a5-46e8-b5e9-bbb508d666be\") " pod="openshift-network-operator/iptables-alerter-qxm9t" Jul 06 18:59:22.009688 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009391 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-lib-modules\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.009688 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009418 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8833070e-9f43-424a-854f-20cadedd63e5-tmp\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.009688 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009467 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqs2w\" (UniqueName: \"kubernetes.io/projected/8833070e-9f43-424a-854f-20cadedd63e5-kube-api-access-kqs2w\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.009688 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009515 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-multus-conf-dir\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.009688 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009549 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-etc-kubernetes\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.009688 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009597 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dqpk\" (UniqueName: \"kubernetes.io/projected/b9d924cf-a6f6-47d0-adae-989ebcab4a23-kube-api-access-2dqpk\") pod \"node-ca-xn6qm\" (UID: \"b9d924cf-a6f6-47d0-adae-989ebcab4a23\") " pod="openshift-image-registry/node-ca-xn6qm" Jul 06 18:59:22.009688 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009633 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30b42dfe-0e9e-441c-a4d8-c5db4383b157-os-release\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.009688 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009657 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-cni-bin\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.009688 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009681 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-os-release\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.010185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009725 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-var-lib-kubelet\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.010185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009740 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-etc-openvswitch\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.010185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009754 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c-agent-certs\") pod \"konnectivity-agent-2lj55\" (UID: \"d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c\") " pod="kube-system/konnectivity-agent-2lj55" Jul 06 18:59:22.010185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009784 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1527a401-f8a5-46e8-b5e9-bbb508d666be-host-slash\") pod \"iptables-alerter-qxm9t\" (UID: \"1527a401-f8a5-46e8-b5e9-bbb508d666be\") " pod="openshift-network-operator/iptables-alerter-qxm9t" Jul 06 18:59:22.010185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009813 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-var-lib-kubelet\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.010185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009835 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8833070e-9f43-424a-854f-20cadedd63e5-etc-tuned\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.010185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009866 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-systemd-units\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.010185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009911 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-cni-netd\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.010185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009943 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9xrf\" (UniqueName: \"kubernetes.io/projected/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-kube-api-access-h9xrf\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.010185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009970 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq5xv\" (UniqueName: \"kubernetes.io/projected/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-kube-api-access-rq5xv\") pod \"network-metrics-daemon-2hf6t\" (UID: \"cd62ad65-3e90-46f3-bdb6-67b9cb3bd158\") " pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:22.010185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.009995 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-run-multus-certs\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.010185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010029 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-run-netns\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.010185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010063 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-multus-daemon-config\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.010185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010089 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-host\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.010185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010111 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-run-openvswitch\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.010185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010133 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-run-ovn\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.010915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010155 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-ovn-node-metrics-cert\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.010915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010179 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30b42dfe-0e9e-441c-a4d8-c5db4383b157-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.010915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010236 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30b42dfe-0e9e-441c-a4d8-c5db4383b157-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.010915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010264 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-multus-socket-dir-parent\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.010915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010287 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-hostroot\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.010915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010309 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-sysconfig\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.010915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010340 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-modprobe-d\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.010915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010366 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-log-socket\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.010915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010391 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30b42dfe-0e9e-441c-a4d8-c5db4383b157-system-cni-dir\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.010915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010415 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-slash\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.010915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010452 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-ovnkube-script-lib\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.010915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010475 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-var-lib-cni-multus\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.010915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010495 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-run\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.010915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010517 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9d924cf-a6f6-47d0-adae-989ebcab4a23-host\") pod \"node-ca-xn6qm\" (UID: \"b9d924cf-a6f6-47d0-adae-989ebcab4a23\") " pod="openshift-image-registry/node-ca-xn6qm" Jul 06 18:59:22.010915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010553 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30b42dfe-0e9e-441c-a4d8-c5db4383b157-cnibin\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.010915 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010587 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvz9n\" (UniqueName: \"kubernetes.io/projected/30b42dfe-0e9e-441c-a4d8-c5db4383b157-kube-api-access-hvz9n\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.011783 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010681 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-run-systemd\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.011783 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010726 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvhss\" (UniqueName: \"kubernetes.io/projected/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-kube-api-access-qvhss\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.011783 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010760 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-sysctl-conf\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.011783 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010786 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30b42dfe-0e9e-441c-a4d8-c5db4383b157-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.011783 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010813 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.011783 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010839 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-ovnkube-config\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.011783 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010866 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-kubelet\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.011783 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.010904 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1527a401-f8a5-46e8-b5e9-bbb508d666be-iptables-alerter-script\") pod \"iptables-alerter-qxm9t\" (UID: \"1527a401-f8a5-46e8-b5e9-bbb508d666be\") " pod="openshift-network-operator/iptables-alerter-qxm9t" Jul 06 18:59:22.013240 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.013219 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 18:59:22.044175 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.044139 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w7ggg" Jul 06 18:59:22.054685 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.054659 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w7ggg" Jul 06 18:59:22.070103 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.070083 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:59:22.096710 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.096677 2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 18:59:22.111366 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.111333 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-kubelet\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.111541 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.111454 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-kubelet\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.111688 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.111656 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1527a401-f8a5-46e8-b5e9-bbb508d666be-iptables-alerter-script\") pod \"iptables-alerter-qxm9t\" (UID: \"1527a401-f8a5-46e8-b5e9-bbb508d666be\") " pod="openshift-network-operator/iptables-alerter-qxm9t" Jul 06 18:59:22.111890 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.111870 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/04e067b8-9d48-42c6-8cac-0254ea3a4328-kubelet-config\") pod \"global-pull-secret-syncer-k9pnm\" (UID: \"04e067b8-9d48-42c6-8cac-0254ea3a4328\") " pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:22.111979 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.111909 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b9d924cf-a6f6-47d0-adae-989ebcab4a23-serviceca\") pod \"node-ca-xn6qm\" (UID: \"b9d924cf-a6f6-47d0-adae-989ebcab4a23\") " pod="openshift-image-registry/node-ca-xn6qm" Jul 06 18:59:22.111979 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.111934 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv88d\" (UniqueName: \"kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d\") pod \"network-check-target-69gb8\" (UID: \"794f47cc-dce2-43e8-b571-d431f4b6cec0\") " pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:22.111979 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.111966 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-cnibin\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.112131 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.111996 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-run-k8s-cni-cncf-io\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.112131 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.112020 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-kubernetes\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.112131 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.112059 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-env-overrides\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.112131 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.112089 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-cni-binary-copy\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.112487 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.112460 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1527a401-f8a5-46e8-b5e9-bbb508d666be-iptables-alerter-script\") pod \"iptables-alerter-qxm9t\" (UID: \"1527a401-f8a5-46e8-b5e9-bbb508d666be\") " pod="openshift-network-operator/iptables-alerter-qxm9t" Jul 06 18:59:22.112610 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.112466 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b9d924cf-a6f6-47d0-adae-989ebcab4a23-serviceca\") pod \"node-ca-xn6qm\" (UID: \"b9d924cf-a6f6-47d0-adae-989ebcab4a23\") " pod="openshift-image-registry/node-ca-xn6qm" Jul 06 18:59:22.112610 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.112594 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-kubernetes\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.112728 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.112648 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-run-k8s-cni-cncf-io\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.112728 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.112695 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-cni-binary-copy\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.112728 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.112716 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-cnibin\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.112891 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.112867 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-var-lib-cni-bin\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.112986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.112927 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-systemd\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.112986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.112961 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret\") pod \"global-pull-secret-syncer-k9pnm\" (UID: \"04e067b8-9d48-42c6-8cac-0254ea3a4328\") " pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:22.113078 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.112994 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30b42dfe-0e9e-441c-a4d8-c5db4383b157-cni-binary-copy\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.113078 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113028 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs\") pod \"network-metrics-daemon-2hf6t\" (UID: \"cd62ad65-3e90-46f3-bdb6-67b9cb3bd158\") " pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:22.113078 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113053 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.113215 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113081 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-system-cni-dir\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.113215 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113110 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-multus-cni-dir\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.113215 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113139 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-sysctl-d\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.113215 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113158 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-env-overrides\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.113215 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113166 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-sys\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.113215 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113190 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-var-lib-openvswitch\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.113483 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113218 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-node-log\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.113483 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113246 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-run-netns\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.113483 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113276 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c-konnectivity-ca\") pod \"konnectivity-agent-2lj55\" (UID: \"d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c\") " pod="kube-system/konnectivity-agent-2lj55" Jul 06 18:59:22.113483 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113278 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-var-lib-cni-bin\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.113483 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113302 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6rkm\" (UniqueName: \"kubernetes.io/projected/1527a401-f8a5-46e8-b5e9-bbb508d666be-kube-api-access-g6rkm\") pod \"iptables-alerter-qxm9t\" (UID: \"1527a401-f8a5-46e8-b5e9-bbb508d666be\") " pod="openshift-network-operator/iptables-alerter-qxm9t" Jul 06 18:59:22.113483 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113331 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-lib-modules\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.113483 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113341 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-systemd\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.113483 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113362 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8833070e-9f43-424a-854f-20cadedd63e5-tmp\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.113827 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113682 2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 18:59:22.115166 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113899 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30b42dfe-0e9e-441c-a4d8-c5db4383b157-cni-binary-copy\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.115166 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.113980 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-node-log\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.115166 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.114035 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.115166 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.114097 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-sys\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.115166 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.114153 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-var-lib-openvswitch\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.115166 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.114179 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-sysctl-d\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.115166 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.114215 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-system-cni-dir\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.115166 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.114336 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-multus-cni-dir\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.115166 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.114415 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:59:22.115166 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.114527 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs podName:cd62ad65-3e90-46f3-bdb6-67b9cb3bd158 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:22.614481146 +0000 UTC m=+2.975644936 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs") pod "network-metrics-daemon-2hf6t" (UID: "cd62ad65-3e90-46f3-bdb6-67b9cb3bd158") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:59:22.115166 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.114719 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c-konnectivity-ca\") pod \"konnectivity-agent-2lj55\" (UID: \"d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c\") " pod="kube-system/konnectivity-agent-2lj55" Jul 06 18:59:22.115166 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.115072 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-lib-modules\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.115166 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.115128 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-run-netns\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.115166 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.115158 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqs2w\" (UniqueName: \"kubernetes.io/projected/8833070e-9f43-424a-854f-20cadedd63e5-kube-api-access-kqs2w\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.115881 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.115207 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-multus-conf-dir\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.115881 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.115232 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-etc-kubernetes\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.115881 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.115262 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dqpk\" (UniqueName: \"kubernetes.io/projected/b9d924cf-a6f6-47d0-adae-989ebcab4a23-kube-api-access-2dqpk\") pod \"node-ca-xn6qm\" (UID: \"b9d924cf-a6f6-47d0-adae-989ebcab4a23\") " pod="openshift-image-registry/node-ca-xn6qm" Jul 06 18:59:22.115881 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.115289 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30b42dfe-0e9e-441c-a4d8-c5db4383b157-os-release\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.115881 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.115316 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-cni-bin\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.116125 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.115955 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-etc-kubernetes\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.116125 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116040 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30b42dfe-0e9e-441c-a4d8-c5db4383b157-os-release\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.116125 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116116 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-cni-bin\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.116310 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116152 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-os-release\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.116310 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116200 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-var-lib-kubelet\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.116310 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116229 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-etc-openvswitch\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.116310 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116262 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c-agent-certs\") pod \"konnectivity-agent-2lj55\" (UID: \"d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c\") " pod="kube-system/konnectivity-agent-2lj55" Jul 06 18:59:22.116310 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116297 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-multus-conf-dir\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.116537 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116391 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1527a401-f8a5-46e8-b5e9-bbb508d666be-host-slash\") pod \"iptables-alerter-qxm9t\" (UID: \"1527a401-f8a5-46e8-b5e9-bbb508d666be\") " pod="openshift-network-operator/iptables-alerter-qxm9t" Jul 06 18:59:22.116920 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116318 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-var-lib-kubelet\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.116920 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116295 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-etc-openvswitch\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.116920 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116204 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-os-release\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.116920 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116735 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-var-lib-kubelet\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.116920 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116753 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1527a401-f8a5-46e8-b5e9-bbb508d666be-host-slash\") pod \"iptables-alerter-qxm9t\" (UID: \"1527a401-f8a5-46e8-b5e9-bbb508d666be\") " pod="openshift-network-operator/iptables-alerter-qxm9t" Jul 06 18:59:22.116920 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116769 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8833070e-9f43-424a-854f-20cadedd63e5-etc-tuned\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.116920 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116797 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-systemd-units\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.116920 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116822 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-cni-netd\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.116920 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116822 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-var-lib-kubelet\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.116920 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116846 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9xrf\" (UniqueName: \"kubernetes.io/projected/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-kube-api-access-h9xrf\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.116920 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116874 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-systemd-units\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.116920 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116891 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-cni-netd\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.116920 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116923 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-device-dir\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116951 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-sys-fs\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.116975 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stvxb\" (UniqueName: \"kubernetes.io/projected/03c38677-11b0-4418-be96-1f112316cf54-kube-api-access-stvxb\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117003 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq5xv\" (UniqueName: \"kubernetes.io/projected/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-kube-api-access-rq5xv\") pod \"network-metrics-daemon-2hf6t\" (UID: \"cd62ad65-3e90-46f3-bdb6-67b9cb3bd158\") " pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117027 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-run-multus-certs\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117052 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-kubelet-dir\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117077 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/777e6e47-55fc-4945-81c1-c3e687f0652b-hosts-file\") pod \"node-resolver-n8wgx\" (UID: \"777e6e47-55fc-4945-81c1-c3e687f0652b\") " pod="openshift-dns/node-resolver-n8wgx" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117100 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-run-netns\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117103 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-run-multus-certs\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117123 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-multus-daemon-config\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117146 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-host\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117168 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m84b\" (UniqueName: \"kubernetes.io/projected/777e6e47-55fc-4945-81c1-c3e687f0652b-kube-api-access-2m84b\") pod \"node-resolver-n8wgx\" (UID: \"777e6e47-55fc-4945-81c1-c3e687f0652b\") " pod="openshift-dns/node-resolver-n8wgx" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117190 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-run-openvswitch\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117191 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-run-netns\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117212 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-run-ovn\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117234 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-ovn-node-metrics-cert\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117256 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-socket-dir\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.117583 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117260 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-host\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117312 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-run-ovn\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117306 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30b42dfe-0e9e-441c-a4d8-c5db4383b157-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117350 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30b42dfe-0e9e-441c-a4d8-c5db4383b157-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117376 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-multus-socket-dir-parent\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117398 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-hostroot\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117422 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-sysconfig\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117469 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30b42dfe-0e9e-441c-a4d8-c5db4383b157-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117473 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-modprobe-d\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117506 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/04e067b8-9d48-42c6-8cac-0254ea3a4328-dbus\") pod \"global-pull-secret-syncer-k9pnm\" (UID: \"04e067b8-9d48-42c6-8cac-0254ea3a4328\") " pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117528 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-registration-dir\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117550 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-modprobe-d\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117605 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-run-openvswitch\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117745 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-multus-daemon-config\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117798 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-hostroot\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117812 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-multus-socket-dir-parent\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117552 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-etc-selinux\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.118357 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117856 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-sysconfig\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117857 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-log-socket\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117889 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-log-socket\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117891 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30b42dfe-0e9e-441c-a4d8-c5db4383b157-system-cni-dir\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117922 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30b42dfe-0e9e-441c-a4d8-c5db4383b157-system-cni-dir\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117959 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-slash\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117924 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-slash\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.117992 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-ovnkube-script-lib\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118027 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-var-lib-cni-multus\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118062 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-run\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118066 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30b42dfe-0e9e-441c-a4d8-c5db4383b157-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118088 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/777e6e47-55fc-4945-81c1-c3e687f0652b-tmp-dir\") pod \"node-resolver-n8wgx\" (UID: \"777e6e47-55fc-4945-81c1-c3e687f0652b\") " pod="openshift-dns/node-resolver-n8wgx" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118118 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-host-var-lib-cni-multus\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118104 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8833070e-9f43-424a-854f-20cadedd63e5-tmp\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118139 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9d924cf-a6f6-47d0-adae-989ebcab4a23-host\") pod \"node-ca-xn6qm\" (UID: \"b9d924cf-a6f6-47d0-adae-989ebcab4a23\") " pod="openshift-image-registry/node-ca-xn6qm" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118146 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-run\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118163 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30b42dfe-0e9e-441c-a4d8-c5db4383b157-cnibin\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118188 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9d924cf-a6f6-47d0-adae-989ebcab4a23-host\") pod \"node-ca-xn6qm\" (UID: \"b9d924cf-a6f6-47d0-adae-989ebcab4a23\") " pod="openshift-image-registry/node-ca-xn6qm" Jul 06 18:59:22.119158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118188 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvz9n\" (UniqueName: \"kubernetes.io/projected/30b42dfe-0e9e-441c-a4d8-c5db4383b157-kube-api-access-hvz9n\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.119986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118224 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-run-systemd\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.119986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118226 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30b42dfe-0e9e-441c-a4d8-c5db4383b157-cnibin\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.119986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118250 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvhss\" (UniqueName: \"kubernetes.io/projected/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-kube-api-access-qvhss\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.119986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118273 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-sysctl-conf\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.119986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118279 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-run-systemd\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.119986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118380 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30b42dfe-0e9e-441c-a4d8-c5db4383b157-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.119986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118395 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8833070e-9f43-424a-854f-20cadedd63e5-etc-sysctl-conf\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.119986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118409 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.119986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118453 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-ovnkube-config\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.119986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118457 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-ovnkube-script-lib\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.119986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118551 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.119986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118859 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30b42dfe-0e9e-441c-a4d8-c5db4383b157-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.119986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.118898 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-ovnkube-config\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.119986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.119324 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c-agent-certs\") pod \"konnectivity-agent-2lj55\" (UID: \"d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c\") " pod="kube-system/konnectivity-agent-2lj55" Jul 06 18:59:22.119986 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.119915 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8833070e-9f43-424a-854f-20cadedd63e5-etc-tuned\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.120866 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.120826 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-ovn-node-metrics-cert\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.125360 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.125337 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:59:22.125464 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.125367 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:59:22.125464 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.125384 2557 projected.go:196] Error preparing data for projected volume kube-api-access-vv88d for pod openshift-network-diagnostics/network-check-target-69gb8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:59:22.125552 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.125475 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d podName:794f47cc-dce2-43e8-b571-d431f4b6cec0 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:22.62545516 +0000 UTC m=+2.986618930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vv88d" (UniqueName: "kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d") pod "network-check-target-69gb8" (UID: "794f47cc-dce2-43e8-b571-d431f4b6cec0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:59:22.132371 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.132345 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvz9n\" (UniqueName: \"kubernetes.io/projected/30b42dfe-0e9e-441c-a4d8-c5db4383b157-kube-api-access-hvz9n\") pod \"multus-additional-cni-plugins-vjmjt\" (UID: \"30b42dfe-0e9e-441c-a4d8-c5db4383b157\") " pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.133002 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.132972 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqs2w\" (UniqueName: \"kubernetes.io/projected/8833070e-9f43-424a-854f-20cadedd63e5-kube-api-access-kqs2w\") pod \"tuned-pg99r\" (UID: \"8833070e-9f43-424a-854f-20cadedd63e5\") " pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.133609 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.133575 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6rkm\" (UniqueName: \"kubernetes.io/projected/1527a401-f8a5-46e8-b5e9-bbb508d666be-kube-api-access-g6rkm\") pod \"iptables-alerter-qxm9t\" (UID: \"1527a401-f8a5-46e8-b5e9-bbb508d666be\") " pod="openshift-network-operator/iptables-alerter-qxm9t" Jul 06 18:59:22.134263 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.134204 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dqpk\" (UniqueName: \"kubernetes.io/projected/b9d924cf-a6f6-47d0-adae-989ebcab4a23-kube-api-access-2dqpk\") pod \"node-ca-xn6qm\" (UID: \"b9d924cf-a6f6-47d0-adae-989ebcab4a23\") " pod="openshift-image-registry/node-ca-xn6qm" Jul 06 18:59:22.134993 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.134946 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq5xv\" (UniqueName: \"kubernetes.io/projected/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-kube-api-access-rq5xv\") pod \"network-metrics-daemon-2hf6t\" (UID: \"cd62ad65-3e90-46f3-bdb6-67b9cb3bd158\") " pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:22.135364 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.135338 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9xrf\" (UniqueName: \"kubernetes.io/projected/02b6efb3-bbdd-4955-95fa-0cebf700b5a3-kube-api-access-h9xrf\") pod \"ovnkube-node-z6w5n\" (UID: \"02b6efb3-bbdd-4955-95fa-0cebf700b5a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.137774 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.137751 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvhss\" (UniqueName: \"kubernetes.io/projected/6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891-kube-api-access-qvhss\") pod \"multus-lsdps\" (UID: \"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891\") " pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.219415 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219374 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/04e067b8-9d48-42c6-8cac-0254ea3a4328-kubelet-config\") pod \"global-pull-secret-syncer-k9pnm\" (UID: \"04e067b8-9d48-42c6-8cac-0254ea3a4328\") " pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:22.219597 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219460 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret\") pod \"global-pull-secret-syncer-k9pnm\" (UID: \"04e067b8-9d48-42c6-8cac-0254ea3a4328\") " pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:22.219597 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219520 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-device-dir\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.219597 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219539 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/04e067b8-9d48-42c6-8cac-0254ea3a4328-kubelet-config\") pod \"global-pull-secret-syncer-k9pnm\" (UID: \"04e067b8-9d48-42c6-8cac-0254ea3a4328\") " pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:22.219597 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.219563 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:59:22.219782 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219606 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-device-dir\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.219782 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219614 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-sys-fs\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.219782 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.219628 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret podName:04e067b8-9d48-42c6-8cac-0254ea3a4328 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:22.719608994 +0000 UTC m=+3.080772780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret") pod "global-pull-secret-syncer-k9pnm" (UID: "04e067b8-9d48-42c6-8cac-0254ea3a4328") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:59:22.219782 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219548 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-sys-fs\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.219782 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219663 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stvxb\" (UniqueName: \"kubernetes.io/projected/03c38677-11b0-4418-be96-1f112316cf54-kube-api-access-stvxb\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.219782 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219693 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-kubelet-dir\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.219782 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219718 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/777e6e47-55fc-4945-81c1-c3e687f0652b-hosts-file\") pod \"node-resolver-n8wgx\" (UID: \"777e6e47-55fc-4945-81c1-c3e687f0652b\") " pod="openshift-dns/node-resolver-n8wgx" Jul 06 18:59:22.219782 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219749 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m84b\" (UniqueName: \"kubernetes.io/projected/777e6e47-55fc-4945-81c1-c3e687f0652b-kube-api-access-2m84b\") pod \"node-resolver-n8wgx\" (UID: \"777e6e47-55fc-4945-81c1-c3e687f0652b\") " pod="openshift-dns/node-resolver-n8wgx" Jul 06 18:59:22.219782 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219776 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-kubelet-dir\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.220176 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219776 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-socket-dir\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.220176 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219834 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/04e067b8-9d48-42c6-8cac-0254ea3a4328-dbus\") pod \"global-pull-secret-syncer-k9pnm\" (UID: \"04e067b8-9d48-42c6-8cac-0254ea3a4328\") " pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:22.220176 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219838 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/777e6e47-55fc-4945-81c1-c3e687f0652b-hosts-file\") pod \"node-resolver-n8wgx\" (UID: \"777e6e47-55fc-4945-81c1-c3e687f0652b\") " pod="openshift-dns/node-resolver-n8wgx" Jul 06 18:59:22.220176 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219872 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-registration-dir\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.220176 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219898 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-etc-selinux\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.220176 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219900 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-socket-dir\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.220176 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219928 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/777e6e47-55fc-4945-81c1-c3e687f0652b-tmp-dir\") pod \"node-resolver-n8wgx\" (UID: \"777e6e47-55fc-4945-81c1-c3e687f0652b\") " pod="openshift-dns/node-resolver-n8wgx" Jul 06 18:59:22.220176 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219960 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-registration-dir\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.220176 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.219986 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/04e067b8-9d48-42c6-8cac-0254ea3a4328-dbus\") pod \"global-pull-secret-syncer-k9pnm\" (UID: \"04e067b8-9d48-42c6-8cac-0254ea3a4328\") " pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:22.220176 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.220041 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/03c38677-11b0-4418-be96-1f112316cf54-etc-selinux\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.220527 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.220254 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/777e6e47-55fc-4945-81c1-c3e687f0652b-tmp-dir\") pod \"node-resolver-n8wgx\" (UID: \"777e6e47-55fc-4945-81c1-c3e687f0652b\") " pod="openshift-dns/node-resolver-n8wgx" Jul 06 18:59:22.232014 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.231923 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stvxb\" (UniqueName: \"kubernetes.io/projected/03c38677-11b0-4418-be96-1f112316cf54-kube-api-access-stvxb\") pod \"aws-ebs-csi-driver-node-txlv5\" (UID: \"03c38677-11b0-4418-be96-1f112316cf54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.232984 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.232958 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m84b\" (UniqueName: \"kubernetes.io/projected/777e6e47-55fc-4945-81c1-c3e687f0652b-kube-api-access-2m84b\") pod \"node-resolver-n8wgx\" (UID: \"777e6e47-55fc-4945-81c1-c3e687f0652b\") " pod="openshift-dns/node-resolver-n8wgx" Jul 06 18:59:22.297456 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.297411 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:22.307526 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.307500 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xn6qm" Jul 06 18:59:22.316121 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.316099 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vjmjt" Jul 06 18:59:22.322909 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.322885 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lsdps" Jul 06 18:59:22.338384 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.338361 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2lj55" Jul 06 18:59:22.347890 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.347872 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qxm9t" Jul 06 18:59:22.357341 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.357321 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pg99r" Jul 06 18:59:22.364864 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.364845 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" Jul 06 18:59:22.370340 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.370313 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n8wgx" Jul 06 18:59:22.622111 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.622072 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs\") pod \"network-metrics-daemon-2hf6t\" (UID: \"cd62ad65-3e90-46f3-bdb6-67b9cb3bd158\") " pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:22.622305 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.622207 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:59:22.622305 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.622269 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs podName:cd62ad65-3e90-46f3-bdb6-67b9cb3bd158 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:23.622256062 +0000 UTC m=+3.983419832 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs") pod "network-metrics-daemon-2hf6t" (UID: "cd62ad65-3e90-46f3-bdb6-67b9cb3bd158") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:59:22.722941 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.722898 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv88d\" (UniqueName: \"kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d\") pod \"network-check-target-69gb8\" (UID: \"794f47cc-dce2-43e8-b571-d431f4b6cec0\") " pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:22.723128 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:22.722958 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret\") pod \"global-pull-secret-syncer-k9pnm\" (UID: \"04e067b8-9d48-42c6-8cac-0254ea3a4328\") " pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:22.723128 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.723083 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:59:22.723128 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.723122 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:59:22.723282 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.723136 2557 projected.go:196] Error preparing data for projected volume kube-api-access-vv88d for pod openshift-network-diagnostics/network-check-target-69gb8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:59:22.723282 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.723084 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:59:22.723282 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.723200 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d podName:794f47cc-dce2-43e8-b571-d431f4b6cec0 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:23.723181219 +0000 UTC m=+4.084344993 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vv88d" (UniqueName: "kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d") pod "network-check-target-69gb8" (UID: "794f47cc-dce2-43e8-b571-d431f4b6cec0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:59:22.723282 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:22.723260 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret podName:04e067b8-9d48-42c6-8cac-0254ea3a4328 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:23.723238017 +0000 UTC m=+4.084401794 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret") pod "global-pull-secret-syncer-k9pnm" (UID: "04e067b8-9d48-42c6-8cac-0254ea3a4328") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:59:23.058116 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:23.058065 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 18:54:22 +0000 UTC" deadline="2028-01-19 14:16:59.581906298 +0000 UTC" Jul 06 18:59:23.058116 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:23.058102 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13483h17m36.523809243s" Jul 06 18:59:23.178534 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:23.178505 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8833070e_9f43_424a_854f_20cadedd63e5.slice/crio-c0c843b6384c0f77e816dccec3cd6dc27126e94030dcc6651773cb4ac878cdaa WatchSource:0}: Error finding container c0c843b6384c0f77e816dccec3cd6dc27126e94030dcc6651773cb4ac878cdaa: Status 404 returned error can't find the container with id c0c843b6384c0f77e816dccec3cd6dc27126e94030dcc6651773cb4ac878cdaa Jul 06 18:59:23.185941 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:23.185917 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cbe1e00_4d4b_4c07_bd9d_93c6a0c70891.slice/crio-5adb257fec1512a8fd7105daf78026fb0d70aa7413f6245d64f9a083c7b7f752 WatchSource:0}: Error finding container 5adb257fec1512a8fd7105daf78026fb0d70aa7413f6245d64f9a083c7b7f752: Status 404 returned error can't find the container with id 5adb257fec1512a8fd7105daf78026fb0d70aa7413f6245d64f9a083c7b7f752 Jul 06 18:59:23.187576 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:23.187553 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02b6efb3_bbdd_4955_95fa_0cebf700b5a3.slice/crio-cfd81fcdec907381c3608a78e26194f99df14d496d660b4cc7bd1fb485504fa7 WatchSource:0}: Error finding container cfd81fcdec907381c3608a78e26194f99df14d496d660b4cc7bd1fb485504fa7: Status 404 returned error can't find the container with id cfd81fcdec907381c3608a78e26194f99df14d496d660b4cc7bd1fb485504fa7 Jul 06 18:59:23.190854 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:23.190410 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:23.190854 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:23.190554 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:23.194808 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:23.194778 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9d924cf_a6f6_47d0_adae_989ebcab4a23.slice/crio-4890c4cef917d2318eca4407088af8a9678559e9af4f868f0c5e4388db1c3e84 WatchSource:0}: Error finding container 4890c4cef917d2318eca4407088af8a9678559e9af4f868f0c5e4388db1c3e84: Status 404 returned error can't find the container with id 4890c4cef917d2318eca4407088af8a9678559e9af4f868f0c5e4388db1c3e84 Jul 06 18:59:23.196697 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:23.196670 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c38677_11b0_4418_be96_1f112316cf54.slice/crio-16c8f6f33ccf0b797d67b31bd2e6a294b1ceba7f2c9e9f913a27eac942985640 WatchSource:0}: Error finding container 16c8f6f33ccf0b797d67b31bd2e6a294b1ceba7f2c9e9f913a27eac942985640: Status 404 returned error can't find the container with id 16c8f6f33ccf0b797d67b31bd2e6a294b1ceba7f2c9e9f913a27eac942985640 Jul 06 18:59:23.197466 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:23.197256 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod777e6e47_55fc_4945_81c1_c3e687f0652b.slice/crio-b2b95e8948f8b7e5883b6ff6124a749971c81eb4ac76c2f5cc058046e25045d2 WatchSource:0}: Error finding container b2b95e8948f8b7e5883b6ff6124a749971c81eb4ac76c2f5cc058046e25045d2: Status 404 returned error can't find the container with id b2b95e8948f8b7e5883b6ff6124a749971c81eb4ac76c2f5cc058046e25045d2 Jul 06 18:59:23.198939 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:23.198919 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1527a401_f8a5_46e8_b5e9_bbb508d666be.slice/crio-b38c7333d7510a3664d4eb81a2d278819d2ce6943a06f03c20c7baa27ac598a0 WatchSource:0}: Error finding container b38c7333d7510a3664d4eb81a2d278819d2ce6943a06f03c20c7baa27ac598a0: Status 404 returned error can't find the container with id b38c7333d7510a3664d4eb81a2d278819d2ce6943a06f03c20c7baa27ac598a0 Jul 06 18:59:23.199911 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:23.199829 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2lj55" event={"ID":"d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c","Type":"ContainerStarted","Data":"ee728f0bffacb7a24bff078312a582269d51dc52dd895a2671e169e53bc93b6b"} Jul 06 18:59:23.199911 ip-10-0-131-230 kubenswrapper[2557]: W0706 18:59:23.199874 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30b42dfe_0e9e_441c_a4d8_c5db4383b157.slice/crio-c677afa3aaf258fbb23028bf3ed1f142444df16c3e9e6de370e02ba3a757eee8 WatchSource:0}: Error finding container c677afa3aaf258fbb23028bf3ed1f142444df16c3e9e6de370e02ba3a757eee8: Status 404 returned error can't find the container with id c677afa3aaf258fbb23028bf3ed1f142444df16c3e9e6de370e02ba3a757eee8 Jul 06 18:59:23.201740 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:23.201635 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lsdps" event={"ID":"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891","Type":"ContainerStarted","Data":"5adb257fec1512a8fd7105daf78026fb0d70aa7413f6245d64f9a083c7b7f752"} Jul 06 18:59:23.202939 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:23.202900 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" event={"ID":"02b6efb3-bbdd-4955-95fa-0cebf700b5a3","Type":"ContainerStarted","Data":"cfd81fcdec907381c3608a78e26194f99df14d496d660b4cc7bd1fb485504fa7"} Jul 06 18:59:23.204088 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:23.204065 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pg99r" event={"ID":"8833070e-9f43-424a-854f-20cadedd63e5","Type":"ContainerStarted","Data":"c0c843b6384c0f77e816dccec3cd6dc27126e94030dcc6651773cb4ac878cdaa"} Jul 06 18:59:23.630707 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:23.630334 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs\") pod \"network-metrics-daemon-2hf6t\" (UID: \"cd62ad65-3e90-46f3-bdb6-67b9cb3bd158\") " pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:23.630879 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:23.630615 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:59:23.630934 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:23.630886 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs podName:cd62ad65-3e90-46f3-bdb6-67b9cb3bd158 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:25.630863056 +0000 UTC m=+5.992026832 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs") pod "network-metrics-daemon-2hf6t" (UID: "cd62ad65-3e90-46f3-bdb6-67b9cb3bd158") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:59:23.732741 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:23.731807 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv88d\" (UniqueName: \"kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d\") pod \"network-check-target-69gb8\" (UID: \"794f47cc-dce2-43e8-b571-d431f4b6cec0\") " pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:23.732741 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:23.731867 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret\") pod \"global-pull-secret-syncer-k9pnm\" (UID: \"04e067b8-9d48-42c6-8cac-0254ea3a4328\") " pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:23.732741 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:23.732056 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:59:23.732741 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:23.732121 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret podName:04e067b8-9d48-42c6-8cac-0254ea3a4328 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:25.73210302 +0000 UTC m=+6.093266794 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret") pod "global-pull-secret-syncer-k9pnm" (UID: "04e067b8-9d48-42c6-8cac-0254ea3a4328") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:59:23.732741 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:23.732540 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:59:23.732741 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:23.732562 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:59:23.732741 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:23.732575 2557 projected.go:196] Error preparing data for projected volume kube-api-access-vv88d for pod openshift-network-diagnostics/network-check-target-69gb8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:59:23.732741 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:23.732623 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d podName:794f47cc-dce2-43e8-b571-d431f4b6cec0 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:25.732607943 +0000 UTC m=+6.093771714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vv88d" (UniqueName: "kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d") pod "network-check-target-69gb8" (UID: "794f47cc-dce2-43e8-b571-d431f4b6cec0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:59:24.058498 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:24.058418 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 18:54:22 +0000 UTC" deadline="2027-12-31 09:00:56.071841769 +0000 UTC" Jul 06 18:59:24.058498 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:24.058496 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13022h1m32.013350397s" Jul 06 18:59:24.192254 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:24.191624 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:24.192254 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:24.191774 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:24.192254 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:24.191627 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:24.192254 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:24.192206 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:24.241006 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:24.240806 2557 generic.go:356] "Generic (PLEG): container finished" podID="924289aee6847fb393cec64c82071a0e" containerID="882c456def51627666851a7a80d97078e339812f2d22ecd31a51ce003ec96960" exitCode=0 Jul 06 18:59:24.241006 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:24.240912 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal" event={"ID":"924289aee6847fb393cec64c82071a0e","Type":"ContainerDied","Data":"882c456def51627666851a7a80d97078e339812f2d22ecd31a51ce003ec96960"} Jul 06 18:59:24.260466 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:24.260412 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-230.ec2.internal" event={"ID":"f78ec814e87d18142aa89255a77769d0","Type":"ContainerStarted","Data":"c0b134d8fdbca4d88d30b2f98f212d151a24ff921e99369cfa2807b7d89c3b13"} Jul 06 18:59:24.273784 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:24.273416 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qxm9t" event={"ID":"1527a401-f8a5-46e8-b5e9-bbb508d666be","Type":"ContainerStarted","Data":"b38c7333d7510a3664d4eb81a2d278819d2ce6943a06f03c20c7baa27ac598a0"} Jul 06 18:59:24.277723 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:24.277664 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-230.ec2.internal" podStartSLOduration=3.27764404 podStartE2EDuration="3.27764404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:24.276948154 +0000 UTC m=+4.638111948" watchObservedRunningTime="2026-07-06 18:59:24.27764404 +0000 UTC m=+4.638807834" Jul 06 18:59:24.291586 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:24.291550 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmjt" event={"ID":"30b42dfe-0e9e-441c-a4d8-c5db4383b157","Type":"ContainerStarted","Data":"c677afa3aaf258fbb23028bf3ed1f142444df16c3e9e6de370e02ba3a757eee8"} Jul 06 18:59:24.310960 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:24.310818 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n8wgx" event={"ID":"777e6e47-55fc-4945-81c1-c3e687f0652b","Type":"ContainerStarted","Data":"b2b95e8948f8b7e5883b6ff6124a749971c81eb4ac76c2f5cc058046e25045d2"} Jul 06 18:59:24.324769 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:24.322788 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" event={"ID":"03c38677-11b0-4418-be96-1f112316cf54","Type":"ContainerStarted","Data":"16c8f6f33ccf0b797d67b31bd2e6a294b1ceba7f2c9e9f913a27eac942985640"} Jul 06 18:59:24.332392 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:24.332305 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xn6qm" event={"ID":"b9d924cf-a6f6-47d0-adae-989ebcab4a23","Type":"ContainerStarted","Data":"4890c4cef917d2318eca4407088af8a9678559e9af4f868f0c5e4388db1c3e84"} Jul 06 18:59:25.190887 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:25.190853 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:25.191382 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:25.191006 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:25.349276 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:25.349239 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal" event={"ID":"924289aee6847fb393cec64c82071a0e","Type":"ContainerStarted","Data":"15714f1a4d35c84e318693b9a1e2a57d9358c56556dc2b5940386d3bec8aeddd"} Jul 06 18:59:25.648496 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:25.648454 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs\") pod \"network-metrics-daemon-2hf6t\" (UID: \"cd62ad65-3e90-46f3-bdb6-67b9cb3bd158\") " pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:25.648691 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:25.648640 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:59:25.648759 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:25.648708 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs podName:cd62ad65-3e90-46f3-bdb6-67b9cb3bd158 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:29.64868731 +0000 UTC m=+10.009851082 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs") pod "network-metrics-daemon-2hf6t" (UID: "cd62ad65-3e90-46f3-bdb6-67b9cb3bd158") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:59:25.749251 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:25.749136 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv88d\" (UniqueName: \"kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d\") pod \"network-check-target-69gb8\" (UID: \"794f47cc-dce2-43e8-b571-d431f4b6cec0\") " pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:25.749251 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:25.749201 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret\") pod \"global-pull-secret-syncer-k9pnm\" (UID: \"04e067b8-9d48-42c6-8cac-0254ea3a4328\") " pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:25.749692 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:25.749385 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:59:25.749692 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:25.749468 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret podName:04e067b8-9d48-42c6-8cac-0254ea3a4328 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:29.749450236 +0000 UTC m=+10.110614021 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret") pod "global-pull-secret-syncer-k9pnm" (UID: "04e067b8-9d48-42c6-8cac-0254ea3a4328") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:59:25.749921 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:25.749896 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:59:25.749921 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:25.749915 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:59:25.750202 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:25.749927 2557 projected.go:196] Error preparing data for projected volume kube-api-access-vv88d for pod openshift-network-diagnostics/network-check-target-69gb8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:59:25.750202 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:25.749972 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d podName:794f47cc-dce2-43e8-b571-d431f4b6cec0 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:29.749957079 +0000 UTC m=+10.111120851 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vv88d" (UniqueName: "kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d") pod "network-check-target-69gb8" (UID: "794f47cc-dce2-43e8-b571-d431f4b6cec0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:59:26.191237 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:26.191202 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:26.191722 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:26.191243 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:26.191722 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:26.191370 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:26.191722 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:26.191507 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:27.190787 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:27.190742 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:27.190974 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:27.190890 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:28.190713 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:28.190570 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:28.190713 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:28.190600 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:28.191229 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:28.190754 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:28.191229 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:28.190858 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:29.190453 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:29.190403 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:29.190631 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:29.190561 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:29.681511 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:29.680965 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs\") pod \"network-metrics-daemon-2hf6t\" (UID: \"cd62ad65-3e90-46f3-bdb6-67b9cb3bd158\") " pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:29.681511 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:29.681099 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:59:29.681511 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:29.681176 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs podName:cd62ad65-3e90-46f3-bdb6-67b9cb3bd158 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:37.681151581 +0000 UTC m=+18.042315406 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs") pod "network-metrics-daemon-2hf6t" (UID: "cd62ad65-3e90-46f3-bdb6-67b9cb3bd158") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:59:29.781755 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:29.781707 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv88d\" (UniqueName: \"kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d\") pod \"network-check-target-69gb8\" (UID: \"794f47cc-dce2-43e8-b571-d431f4b6cec0\") " pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:29.781938 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:29.781774 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret\") pod \"global-pull-secret-syncer-k9pnm\" (UID: \"04e067b8-9d48-42c6-8cac-0254ea3a4328\") " pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:29.781938 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:29.781898 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:59:29.781938 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:29.781919 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:59:29.782095 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:29.781925 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:59:29.782095 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:29.781990 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret podName:04e067b8-9d48-42c6-8cac-0254ea3a4328 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:37.781970307 +0000 UTC m=+18.143134091 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret") pod "global-pull-secret-syncer-k9pnm" (UID: "04e067b8-9d48-42c6-8cac-0254ea3a4328") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:59:29.782095 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:29.781993 2557 projected.go:196] Error preparing data for projected volume kube-api-access-vv88d for pod openshift-network-diagnostics/network-check-target-69gb8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:59:29.782095 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:29.782049 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d podName:794f47cc-dce2-43e8-b571-d431f4b6cec0 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:37.782030397 +0000 UTC m=+18.143194185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vv88d" (UniqueName: "kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d") pod "network-check-target-69gb8" (UID: "794f47cc-dce2-43e8-b571-d431f4b6cec0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:59:30.192205 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:30.192136 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:30.192404 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:30.192268 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:30.192404 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:30.192286 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:30.192404 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:30.192351 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:31.190833 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:31.190800 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:31.191351 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:31.190949 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:32.191407 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:32.191367 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:32.191859 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:32.191539 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:32.191859 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:32.191585 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:32.191859 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:32.191694 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:33.190464 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:33.190211 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:33.190464 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:33.190347 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:34.190899 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:34.190692 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:34.190899 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:34.190692 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:34.190899 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:34.190888 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:34.191472 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:34.191300 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:35.190338 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:35.190303 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:35.190525 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:35.190461 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:36.190376 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:36.190342 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:36.190376 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:36.190362 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:36.190920 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:36.190487 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:36.190920 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:36.190576 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:37.191006 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:37.190969 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:37.191496 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:37.191113 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:37.748887 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:37.748849 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs\") pod \"network-metrics-daemon-2hf6t\" (UID: \"cd62ad65-3e90-46f3-bdb6-67b9cb3bd158\") " pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:37.749087 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:37.749012 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:59:37.749087 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:37.749085 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs podName:cd62ad65-3e90-46f3-bdb6-67b9cb3bd158 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:53.749066364 +0000 UTC m=+34.110230134 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs") pod "network-metrics-daemon-2hf6t" (UID: "cd62ad65-3e90-46f3-bdb6-67b9cb3bd158") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:59:37.850088 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:37.849999 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv88d\" (UniqueName: \"kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d\") pod \"network-check-target-69gb8\" (UID: \"794f47cc-dce2-43e8-b571-d431f4b6cec0\") " pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:37.850088 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:37.850048 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret\") pod \"global-pull-secret-syncer-k9pnm\" (UID: \"04e067b8-9d48-42c6-8cac-0254ea3a4328\") " pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:37.850319 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:37.850175 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:59:37.850319 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:37.850185 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:59:37.850319 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:37.850209 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:59:37.850319 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:37.850222 2557 projected.go:196] Error preparing data for projected volume kube-api-access-vv88d for pod openshift-network-diagnostics/network-check-target-69gb8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:59:37.850319 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:37.850246 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret podName:04e067b8-9d48-42c6-8cac-0254ea3a4328 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:53.850225546 +0000 UTC m=+34.211389333 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret") pod "global-pull-secret-syncer-k9pnm" (UID: "04e067b8-9d48-42c6-8cac-0254ea3a4328") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:59:37.850319 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:37.850278 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d podName:794f47cc-dce2-43e8-b571-d431f4b6cec0 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:53.850261419 +0000 UTC m=+34.211425190 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vv88d" (UniqueName: "kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d") pod "network-check-target-69gb8" (UID: "794f47cc-dce2-43e8-b571-d431f4b6cec0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:59:38.193551 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:38.193480 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:38.193551 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:38.193516 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:38.194032 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:38.193625 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:38.194032 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:38.193736 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:39.190588 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:39.190550 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:39.190774 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:39.190723 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:40.194085 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:40.194057 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:40.194085 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:40.194078 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:40.194617 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:40.194167 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:40.194617 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:40.194300 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:41.190793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:41.190764 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:41.190971 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:41.190875 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:42.191900 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.191102 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:42.191900 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:42.191529 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:42.191900 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.191543 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:42.191900 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:42.191656 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:42.385179 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.384951 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2lj55" event={"ID":"d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c","Type":"ContainerStarted","Data":"a331a038de557b65d7933aa13045e97cb8287e0735329cead06fc95176a5e4bf"} Jul 06 18:59:42.387077 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.387051 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" event={"ID":"02b6efb3-bbdd-4955-95fa-0cebf700b5a3","Type":"ContainerStarted","Data":"753ca9976b355829b5e95dea754e64f995f14acdef072fdfae751603ee360606"} Jul 06 18:59:42.387181 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.387113 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" event={"ID":"02b6efb3-bbdd-4955-95fa-0cebf700b5a3","Type":"ContainerStarted","Data":"9028cc7227e30ca04e9ec0a446d4e579d2f718b05fa8acb7fc8b70119039be19"} Jul 06 18:59:42.387181 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.387152 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" event={"ID":"02b6efb3-bbdd-4955-95fa-0cebf700b5a3","Type":"ContainerStarted","Data":"9b5a4f61389aa8fb732e64093ae5608e4238aa4b57a66657c559801a9d536471"} Jul 06 18:59:42.387272 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.387184 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" event={"ID":"02b6efb3-bbdd-4955-95fa-0cebf700b5a3","Type":"ContainerStarted","Data":"70e0906f48faf6cd9ba6dad6c45e529ba2661900d223fcba5343af4f52631bbb"} Jul 06 18:59:42.388350 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.388331 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xn6qm" event={"ID":"b9d924cf-a6f6-47d0-adae-989ebcab4a23","Type":"ContainerStarted","Data":"801405cfa13b37b09427eb15fe10de4093676f9dcd052e9550b05ecf79ff1669"} Jul 06 18:59:42.389800 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.389776 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lsdps" event={"ID":"6cbe1e00-4d4b-4c07-bd9d-93c6a0c70891","Type":"ContainerStarted","Data":"63dde1dd3e01167472333f45782c1db2dc89a96e43d5f369f6b89c7adb7a6731"} Jul 06 18:59:42.391060 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.391042 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pg99r" event={"ID":"8833070e-9f43-424a-854f-20cadedd63e5","Type":"ContainerStarted","Data":"124f83eeeb3449865ccfc92eaeffbd9d56348ea8d608f43e9710d742bd091161"} Jul 06 18:59:42.392425 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.392402 2557 generic.go:356] "Generic (PLEG): container finished" podID="30b42dfe-0e9e-441c-a4d8-c5db4383b157" containerID="6a6ef20180fcab33381bc70c300726fcea7f077ad8b7264a45fe0261948481e9" exitCode=0 Jul 06 18:59:42.392524 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.392464 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmjt" event={"ID":"30b42dfe-0e9e-441c-a4d8-c5db4383b157","Type":"ContainerDied","Data":"6a6ef20180fcab33381bc70c300726fcea7f077ad8b7264a45fe0261948481e9"} Jul 06 18:59:42.393802 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.393750 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n8wgx" event={"ID":"777e6e47-55fc-4945-81c1-c3e687f0652b","Type":"ContainerStarted","Data":"3b71ae65bcd24a355c706b0beb84842106bdfac0449aefd6af7d5d12ce3a2f5a"} Jul 06 18:59:42.395185 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.395167 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" event={"ID":"03c38677-11b0-4418-be96-1f112316cf54","Type":"ContainerStarted","Data":"1f7cf73682310908d0a0291e67ed23e283fa8f54c250a73e5923f514afaad6df"} Jul 06 18:59:42.401244 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.401206 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-230.ec2.internal" podStartSLOduration=21.401195284 podStartE2EDuration="21.401195284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:25.367139771 +0000 UTC m=+5.728303563" watchObservedRunningTime="2026-07-06 18:59:42.401195284 +0000 UTC m=+22.762359077" Jul 06 18:59:42.401621 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.401597 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2lj55" podStartSLOduration=3.636040383 podStartE2EDuration="22.40158827s" totalImagesPullingTime="18.765547887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:42.400875079 +0000 UTC m=+22.762038872" watchObservedRunningTime="2026-07-06 18:59:42.40158827 +0000 UTC m=+22.762752061" Jul 06 18:59:42.419142 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.419110 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xn6qm" podStartSLOduration=3.656907758 podStartE2EDuration="22.419102752s" totalImagesPullingTime="18.762194994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:42.418952474 +0000 UTC m=+22.780116268" watchObservedRunningTime="2026-07-06 18:59:42.419102752 +0000 UTC m=+22.780266544" Jul 06 18:59:42.463140 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.463031 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-n8wgx" podStartSLOduration=3.703213763 podStartE2EDuration="22.46301438s" totalImagesPullingTime="18.759800617s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:42.462953674 +0000 UTC m=+22.824117466" watchObservedRunningTime="2026-07-06 18:59:42.46301438 +0000 UTC m=+22.824178171" Jul 06 18:59:42.485034 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.484990 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lsdps" podStartSLOduration=3.654167575 podStartE2EDuration="22.484974502s" totalImagesPullingTime="18.830806927s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:42.484902995 +0000 UTC m=+22.846066788" watchObservedRunningTime="2026-07-06 18:59:42.484974502 +0000 UTC m=+22.846138293" Jul 06 18:59:42.510169 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.510122 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pg99r" podStartSLOduration=3.730417486 podStartE2EDuration="22.510104767s" totalImagesPullingTime="18.779687281s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:42.509816312 +0000 UTC m=+22.870980098" watchObservedRunningTime="2026-07-06 18:59:42.510104767 +0000 UTC m=+22.871268558" Jul 06 18:59:42.536904 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.536873 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:42.615363 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.615328 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:42.714687 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:42.714540 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:43.190760 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:43.190731 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:43.190945 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:43.190843 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:43.401358 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:43.401324 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" event={"ID":"02b6efb3-bbdd-4955-95fa-0cebf700b5a3","Type":"ContainerStarted","Data":"e8e01d7e2fa1e4e4f3ecd9b26df27e432d1fbd6004413dfe7b34edcb4a1353a2"} Jul 06 18:59:43.401982 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:43.401420 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" event={"ID":"02b6efb3-bbdd-4955-95fa-0cebf700b5a3","Type":"ContainerStarted","Data":"d08f5cb3a3e6301c87454366979d57013683688232b4402076cc43f5337fd8c9"} Jul 06 18:59:43.404287 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:43.404254 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qxm9t" event={"ID":"1527a401-f8a5-46e8-b5e9-bbb508d666be","Type":"ContainerStarted","Data":"96fd608d8470991bb0ee4b172fb41ad0a1f24be6c15cb3a9c3e229a7c99165c0"} Jul 06 18:59:43.496014 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:43.495961 2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 18:59:43.586870 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:43.586817 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:43.724837 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:43.724741 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:44.022515 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:44.022137 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2lj55" Jul 06 18:59:44.022897 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:44.022881 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2lj55" Jul 06 18:59:44.057586 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:44.057530 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qxm9t" podStartSLOduration=5.299460255 podStartE2EDuration="24.057507299s" totalImagesPullingTime="18.758047044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:43.429043227 +0000 UTC m=+23.790207019" watchObservedRunningTime="2026-07-06 18:59:44.057507299 +0000 UTC m=+24.418671086" Jul 06 18:59:44.070067 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:44.069961 2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T18:59:43.495983535Z","UUID":"549be51b-597d-47bb-93b4-fa7ee1a7db9f","Handler":null,"Name":"","Endpoint":""} Jul 06 18:59:44.071522 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:44.071503 2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 18:59:44.071654 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:44.071539 2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 18:59:44.191237 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:44.191204 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:44.191393 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:44.191208 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:44.191393 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:44.191342 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:44.191494 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:44.191410 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:44.410228 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:44.410095 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" event={"ID":"03c38677-11b0-4418-be96-1f112316cf54","Type":"ContainerStarted","Data":"78b2d0a77e9128775842382c7b036f1f9b5bb2ce476da358cd78c64d2f637767"} Jul 06 18:59:44.748977 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:44.748942 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:44.861369 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:44.861337 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:45.191160 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:45.191126 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:45.191405 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:45.191381 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:45.414882 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:45.414849 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" event={"ID":"03c38677-11b0-4418-be96-1f112316cf54","Type":"ContainerStarted","Data":"ae1012dd19cf03a4040cd3780c51ec18cbd655d59069e624f017d7e716c79de1"} Jul 06 18:59:45.418116 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:45.418089 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" event={"ID":"02b6efb3-bbdd-4955-95fa-0cebf700b5a3","Type":"ContainerStarted","Data":"83dc48c824104b2b2d5172682de7bc3b00ed289105908c91149118634bd4ae28"} Jul 06 18:59:45.418246 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:45.418123 2557 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 18:59:45.445793 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:45.445702 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-txlv5" podStartSLOduration=4.413394867 podStartE2EDuration="25.445687475s" totalImagesPullingTime="21.032292608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:45.445513556 +0000 UTC m=+25.806677349" watchObservedRunningTime="2026-07-06 18:59:45.445687475 +0000 UTC m=+25.806851270" Jul 06 18:59:45.497976 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:45.497936 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:45.666010 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:45.665979 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:45.768263 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:45.768086 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:45.869131 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:45.869092 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:46.191288 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:46.191211 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:46.191476 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:46.191330 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:46.191476 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:46.191388 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:46.191570 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:46.191505 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:46.229367 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:46.229335 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:46.295127 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:46.295092 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:46.416267 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:46.416237 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:46.536628 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:46.536593 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:46.602605 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:46.602575 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:46.837053 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:46.837017 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n8wgx_777e6e47-55fc-4945-81c1-c3e687f0652b/dns-node-resolver/0.log" Jul 06 18:59:46.932486 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:46.932449 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:47.018490 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:47.018457 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:47.191100 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:47.191023 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:47.191263 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:47.191151 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:48.161088 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:48.160924 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:48.190840 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:48.190815 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:48.190974 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:48.190949 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:48.191034 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:48.190998 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:48.191127 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:48.191106 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:48.226903 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:48.226876 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xn6qm_b9d924cf-a6f6-47d0-adae-989ebcab4a23/node-ca/0.log" Jul 06 18:59:48.266476 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:48.266422 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:48.427368 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:48.427278 2557 generic.go:356] "Generic (PLEG): container finished" podID="30b42dfe-0e9e-441c-a4d8-c5db4383b157" containerID="00158d0d66921d7e2ec457cf50069147465b8b66b662fae687c975a8e2b71882" exitCode=0 Jul 06 18:59:48.427368 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:48.427338 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmjt" event={"ID":"30b42dfe-0e9e-441c-a4d8-c5db4383b157","Type":"ContainerDied","Data":"00158d0d66921d7e2ec457cf50069147465b8b66b662fae687c975a8e2b71882"} Jul 06 18:59:48.428287 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:48.428128 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2lj55" Jul 06 18:59:48.428287 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:48.428215 2557 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 18:59:48.428800 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:48.428764 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2lj55" Jul 06 18:59:48.431190 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:48.431168 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" event={"ID":"02b6efb3-bbdd-4955-95fa-0cebf700b5a3","Type":"ContainerStarted","Data":"967ca0ffd9aa6562a06bf14079471731de8307ee06043f27de783fe459fd3df8"} Jul 06 18:59:48.431426 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:48.431411 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:48.447965 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:48.447938 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:48.543830 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:48.543773 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" podStartSLOduration=9.708709102 podStartE2EDuration="28.54375603s" totalImagesPullingTime="18.835046928s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:48.5432796 +0000 UTC m=+28.904443390" watchObservedRunningTime="2026-07-06 18:59:48.54375603 +0000 UTC m=+28.904919821" Jul 06 18:59:48.586979 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:48.586939 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:49.087731 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:49.087683 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:49.190268 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:49.190237 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:49.190745 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:49.190343 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:49.434697 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:49.434607 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:49.434697 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:49.434689 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:49.449890 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:49.449868 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 18:59:49.976333 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:49.976261 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k9pnm"] Jul 06 18:59:49.976492 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:49.976368 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:49.976492 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:49.976469 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:49.979669 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:49.979634 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-69gb8"] Jul 06 18:59:49.979823 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:49.979757 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:49.979881 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:49.979856 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:49.980313 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:49.980290 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2hf6t"] Jul 06 18:59:49.980413 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:49.980401 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:49.980548 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:49.980529 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:50.235427 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:50.235334 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:50.438654 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:50.438477 2557 generic.go:356] "Generic (PLEG): container finished" podID="30b42dfe-0e9e-441c-a4d8-c5db4383b157" containerID="69cc4bee8b322877ee7304bd88dfaf3a6ea2981a4f25078f90cb0f2819971b8f" exitCode=0 Jul 06 18:59:50.438654 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:50.438564 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmjt" event={"ID":"30b42dfe-0e9e-441c-a4d8-c5db4383b157","Type":"ContainerDied","Data":"69cc4bee8b322877ee7304bd88dfaf3a6ea2981a4f25078f90cb0f2819971b8f"} Jul 06 18:59:51.190681 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:51.190599 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:51.190837 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:51.190742 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:51.847235 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:51.847140 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:52.190716 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:52.190620 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:52.190872 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:52.190738 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:52.190872 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:52.190789 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:52.190992 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:52.190876 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:52.445215 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:52.445131 2557 generic.go:356] "Generic (PLEG): container finished" podID="30b42dfe-0e9e-441c-a4d8-c5db4383b157" containerID="d48e7da7a5de550391a451103f98e774b7bf0726098c0dc35163a0e3b9af1a4b" exitCode=0 Jul 06 18:59:52.445215 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:52.445195 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmjt" event={"ID":"30b42dfe-0e9e-441c-a4d8-c5db4383b157","Type":"ContainerDied","Data":"d48e7da7a5de550391a451103f98e774b7bf0726098c0dc35163a0e3b9af1a4b"} Jul 06 18:59:52.699006 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:52.698919 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:52.856261 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:52.856170 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:53.034216 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:53.034176 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:53.124862 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:53.124826 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:53.190517 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:53.190482 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:53.190681 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:53.190646 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:53.311211 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:53.311116 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:53.418970 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:53.418931 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:53.616353 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:53.616256 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:53.764406 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:53.764364 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs\") pod \"network-metrics-daemon-2hf6t\" (UID: \"cd62ad65-3e90-46f3-bdb6-67b9cb3bd158\") " pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:53.764582 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:53.764528 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:59:53.764631 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:53.764594 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs podName:cd62ad65-3e90-46f3-bdb6-67b9cb3bd158 nodeName:}" failed. No retries permitted until 2026-07-06 19:00:25.764579627 +0000 UTC m=+66.125743396 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs") pod "network-metrics-daemon-2hf6t" (UID: "cd62ad65-3e90-46f3-bdb6-67b9cb3bd158") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:59:53.783125 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:53.783085 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:53.865137 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:53.865098 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv88d\" (UniqueName: \"kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d\") pod \"network-check-target-69gb8\" (UID: \"794f47cc-dce2-43e8-b571-d431f4b6cec0\") " pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:53.865137 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:53.865144 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret\") pod \"global-pull-secret-syncer-k9pnm\" (UID: \"04e067b8-9d48-42c6-8cac-0254ea3a4328\") " pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:53.865601 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:53.865259 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:59:53.865601 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:53.865275 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:59:53.865601 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:53.865296 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:59:53.865601 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:53.865306 2557 projected.go:196] Error preparing data for projected volume kube-api-access-vv88d for pod openshift-network-diagnostics/network-check-target-69gb8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:59:53.865601 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:53.865316 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret podName:04e067b8-9d48-42c6-8cac-0254ea3a4328 nodeName:}" failed. No retries permitted until 2026-07-06 19:00:25.86530286 +0000 UTC m=+66.226466630 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret") pod "global-pull-secret-syncer-k9pnm" (UID: "04e067b8-9d48-42c6-8cac-0254ea3a4328") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:59:53.865601 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:53.865345 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d podName:794f47cc-dce2-43e8-b571-d431f4b6cec0 nodeName:}" failed. No retries permitted until 2026-07-06 19:00:25.865333984 +0000 UTC m=+66.226497755 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vv88d" (UniqueName: "kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d") pod "network-check-target-69gb8" (UID: "794f47cc-dce2-43e8-b571-d431f4b6cec0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:59:54.193800 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:54.193770 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:54.194065 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:54.193777 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:54.194065 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:54.193874 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:54.194065 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:54.193980 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:54.779489 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:54.779432 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:55.177456 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:55.177350 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:55.191158 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:55.191129 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:55.191298 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:55.191271 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:55.378762 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:55.378723 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:55.574473 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:55.574431 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:55.779266 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:55.779219 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:56.190325 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:56.190294 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:56.190828 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:56.190294 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:56.190828 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:56.190449 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:56.190828 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:56.190532 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:56.413936 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:56.413893 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:56.637832 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:56.637777 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:56.778621 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:56.778584 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:56.977923 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:56.977831 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:57.190902 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:57.190866 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:57.191351 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:57.190989 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:57.374988 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:57.374949 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:57.480625 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:57.480593 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:57.577622 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:57.577583 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:58.037319 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:58.037278 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:58.193740 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:58.193712 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 18:59:58.194123 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:58.193712 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 18:59:58.194123 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:58.193814 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 18:59:58.194123 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:58.193875 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 18:59:58.405292 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:58.405198 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:58.462001 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:58.461964 2557 generic.go:356] "Generic (PLEG): container finished" podID="30b42dfe-0e9e-441c-a4d8-c5db4383b157" containerID="7f696fe17148191b9b1b3b4f59344bea6acd821cdd182ca0438640fa64a53e4b" exitCode=0 Jul 06 18:59:58.462001 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:58.462005 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmjt" event={"ID":"30b42dfe-0e9e-441c-a4d8-c5db4383b157","Type":"ContainerDied","Data":"7f696fe17148191b9b1b3b4f59344bea6acd821cdd182ca0438640fa64a53e4b"} Jul 06 18:59:59.030393 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:59.030353 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:59.183331 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:59.183300 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:59.190832 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:59.190811 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 18:59:59.190930 ip-10-0-131-230 kubenswrapper[2557]: E0706 18:59:59.190913 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 18:59:59.382036 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:59.381933 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:59.466929 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:59.466895 2557 generic.go:356] "Generic (PLEG): container finished" podID="30b42dfe-0e9e-441c-a4d8-c5db4383b157" containerID="7b359c055d2dfabd7651d4a202b9f932b62c079d3b9e4ae4180e9036b33336f2" exitCode=0 Jul 06 18:59:59.467083 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:59.466959 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmjt" event={"ID":"30b42dfe-0e9e-441c-a4d8-c5db4383b157","Type":"ContainerDied","Data":"7b359c055d2dfabd7651d4a202b9f932b62c079d3b9e4ae4180e9036b33336f2"} Jul 06 18:59:59.780850 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:59.780660 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:59.906194 ip-10-0-131-230 kubenswrapper[2557]: I0706 18:59:59.906159 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:00.002447 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:00.002409 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:00.073071 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:00.072983 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:00.191391 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:00.191361 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 19:00:00.191581 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:00.191485 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 19:00:00.191581 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:00.191537 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 19:00:00.191671 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:00.191622 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 19:00:00.402851 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:00.402763 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:00.475619 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:00.475578 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmjt" event={"ID":"30b42dfe-0e9e-441c-a4d8-c5db4383b157","Type":"ContainerStarted","Data":"8902ccb771cdc7f6be0cb77885356b69b93bd76c8058f8ddfd508fd1b3449c57"} Jul 06 19:00:00.502474 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:00.502396 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vjmjt" podStartSLOduration=8.806528892 podStartE2EDuration="40.50238067s" totalImagesPullingTime="31.695851778s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:00.502117352 +0000 UTC m=+40.863281147" watchObservedRunningTime="2026-07-06 19:00:00.50238067 +0000 UTC m=+40.863544461" Jul 06 19:00:00.693381 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:00.693293 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:00.794692 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:00.794662 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:01.190240 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:01.190206 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 19:00:01.190412 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:01.190330 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 19:00:01.617632 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:01.617590 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:01.925657 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:01.925557 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:02.049854 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:02.049824 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:02.190367 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:02.190273 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 19:00:02.190555 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:02.190273 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 19:00:02.190555 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:02.190384 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 19:00:02.190555 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:02.190492 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 19:00:02.314203 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:02.314169 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:02.401843 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:02.401810 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:02.782135 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:02.782098 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:03.190782 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:03.190694 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 19:00:03.190933 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:03.190819 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 19:00:03.560972 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:03.560934 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:03.638015 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:03.637980 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:04.190420 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:04.190386 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 19:00:04.190818 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:04.190386 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 19:00:04.190818 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:04.190567 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 19:00:04.190818 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:04.190677 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 19:00:05.190432 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:05.190395 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 19:00:05.190824 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:05.190521 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 19:00:06.190222 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:06.190196 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 19:00:06.190399 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:06.190196 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 19:00:06.190399 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:06.190300 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 19:00:06.190399 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:06.190370 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 19:00:07.190210 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:07.190178 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 19:00:07.190390 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:07.190285 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 19:00:08.190709 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:08.190672 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 19:00:08.191075 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:08.190816 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 19:00:08.191075 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:08.190863 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 19:00:08.191075 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:08.190952 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 19:00:09.191005 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:09.190968 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 19:00:09.191387 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:09.191093 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 19:00:10.191973 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:10.191934 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 19:00:10.192482 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:10.192055 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 19:00:10.192482 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:10.192088 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 19:00:10.192482 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:10.192180 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 19:00:11.190672 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:11.190638 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 19:00:11.190896 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:11.190751 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hf6t" podUID="cd62ad65-3e90-46f3-bdb6-67b9cb3bd158" Jul 06 19:00:12.190294 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:12.190263 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 19:00:12.190781 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:12.190372 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69gb8" podUID="794f47cc-dce2-43e8-b571-d431f4b6cec0" Jul 06 19:00:12.190781 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:12.190415 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 19:00:12.190781 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:12.190481 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9pnm" podUID="04e067b8-9d48-42c6-8cac-0254ea3a4328" Jul 06 19:00:12.267878 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:12.267843 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:12.404195 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:12.404161 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:12.970923 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:12.970896 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-230.ec2.internal" event="NodeReady" Jul 06 19:00:12.971170 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:12.971037 2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 19:00:13.101167 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.101137 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8sq64"] Jul 06 19:00:13.123595 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.123561 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ptnmn"] Jul 06 19:00:13.123749 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.123687 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8sq64" Jul 06 19:00:13.129033 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.129006 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 19:00:13.129810 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.129776 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 19:00:13.131481 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.131453 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 19:00:13.138427 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.138403 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8sq64"] Jul 06 19:00:13.138427 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.138427 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-r2qkq\"" Jul 06 19:00:13.138590 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.138453 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ptnmn"] Jul 06 19:00:13.138590 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.138526 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ptnmn" Jul 06 19:00:13.143697 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.143676 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 19:00:13.143831 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.143716 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 19:00:13.143831 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.143714 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2f7gj\"" Jul 06 19:00:13.191024 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.190991 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 19:00:13.193949 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.193926 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:00:13.194755 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.194737 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6kw7z\"" Jul 06 19:00:13.200382 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.200363 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3532d5a1-91d6-426c-8e98-a6afaaec3c8e-config-volume\") pod \"dns-default-ptnmn\" (UID: \"3532d5a1-91d6-426c-8e98-a6afaaec3c8e\") " pod="openshift-dns/dns-default-ptnmn" Jul 06 19:00:13.200478 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.200390 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3532d5a1-91d6-426c-8e98-a6afaaec3c8e-tmp-dir\") pod \"dns-default-ptnmn\" (UID: \"3532d5a1-91d6-426c-8e98-a6afaaec3c8e\") " pod="openshift-dns/dns-default-ptnmn" Jul 06 19:00:13.200522 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.200483 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpngw\" (UniqueName: \"kubernetes.io/projected/3532d5a1-91d6-426c-8e98-a6afaaec3c8e-kube-api-access-rpngw\") pod \"dns-default-ptnmn\" (UID: \"3532d5a1-91d6-426c-8e98-a6afaaec3c8e\") " pod="openshift-dns/dns-default-ptnmn" Jul 06 19:00:13.200559 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.200520 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3532d5a1-91d6-426c-8e98-a6afaaec3c8e-metrics-tls\") pod \"dns-default-ptnmn\" (UID: \"3532d5a1-91d6-426c-8e98-a6afaaec3c8e\") " pod="openshift-dns/dns-default-ptnmn" Jul 06 19:00:13.200559 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.200548 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5nt4\" (UniqueName: \"kubernetes.io/projected/b7605339-9208-4293-b7f5-0573dd8d65de-kube-api-access-q5nt4\") pod \"ingress-canary-8sq64\" (UID: \"b7605339-9208-4293-b7f5-0573dd8d65de\") " pod="openshift-ingress-canary/ingress-canary-8sq64" Jul 06 19:00:13.200631 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.200580 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7605339-9208-4293-b7f5-0573dd8d65de-cert\") pod \"ingress-canary-8sq64\" (UID: \"b7605339-9208-4293-b7f5-0573dd8d65de\") " pod="openshift-ingress-canary/ingress-canary-8sq64" Jul 06 19:00:13.246712 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.246676 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-q725j"] Jul 06 19:00:13.260357 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.260330 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.265836 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.265810 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 19:00:13.271898 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.271877 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 19:00:13.276281 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.276261 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 19:00:13.278469 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.278428 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-g47v9\"" Jul 06 19:00:13.278926 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.278900 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q725j"] Jul 06 19:00:13.282633 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.282615 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 19:00:13.301116 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.301091 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd8fd2e7-6baa-43ec-85c7-457e042c3834-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q725j\" (UID: \"dd8fd2e7-6baa-43ec-85c7-457e042c3834\") " pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.301228 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.301135 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpngw\" (UniqueName: \"kubernetes.io/projected/3532d5a1-91d6-426c-8e98-a6afaaec3c8e-kube-api-access-rpngw\") pod \"dns-default-ptnmn\" (UID: \"3532d5a1-91d6-426c-8e98-a6afaaec3c8e\") " pod="openshift-dns/dns-default-ptnmn" Jul 06 19:00:13.301228 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.301167 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3532d5a1-91d6-426c-8e98-a6afaaec3c8e-metrics-tls\") pod \"dns-default-ptnmn\" (UID: \"3532d5a1-91d6-426c-8e98-a6afaaec3c8e\") " pod="openshift-dns/dns-default-ptnmn" Jul 06 19:00:13.301228 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.301196 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dd8fd2e7-6baa-43ec-85c7-457e042c3834-crio-socket\") pod \"insights-runtime-extractor-q725j\" (UID: \"dd8fd2e7-6baa-43ec-85c7-457e042c3834\") " pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.301228 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.301225 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5nt4\" (UniqueName: \"kubernetes.io/projected/b7605339-9208-4293-b7f5-0573dd8d65de-kube-api-access-q5nt4\") pod \"ingress-canary-8sq64\" (UID: \"b7605339-9208-4293-b7f5-0573dd8d65de\") " pod="openshift-ingress-canary/ingress-canary-8sq64" Jul 06 19:00:13.301379 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.301249 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd8fd2e7-6baa-43ec-85c7-457e042c3834-data-volume\") pod \"insights-runtime-extractor-q725j\" (UID: \"dd8fd2e7-6baa-43ec-85c7-457e042c3834\") " pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.301379 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.301292 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7605339-9208-4293-b7f5-0573dd8d65de-cert\") pod \"ingress-canary-8sq64\" (UID: \"b7605339-9208-4293-b7f5-0573dd8d65de\") " pod="openshift-ingress-canary/ingress-canary-8sq64" Jul 06 19:00:13.301379 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.301322 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtlqr\" (UniqueName: \"kubernetes.io/projected/dd8fd2e7-6baa-43ec-85c7-457e042c3834-kube-api-access-vtlqr\") pod \"insights-runtime-extractor-q725j\" (UID: \"dd8fd2e7-6baa-43ec-85c7-457e042c3834\") " pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.301379 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.301353 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd8fd2e7-6baa-43ec-85c7-457e042c3834-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q725j\" (UID: \"dd8fd2e7-6baa-43ec-85c7-457e042c3834\") " pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.301561 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.301382 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3532d5a1-91d6-426c-8e98-a6afaaec3c8e-config-volume\") pod \"dns-default-ptnmn\" (UID: \"3532d5a1-91d6-426c-8e98-a6afaaec3c8e\") " pod="openshift-dns/dns-default-ptnmn" Jul 06 19:00:13.301561 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.301417 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3532d5a1-91d6-426c-8e98-a6afaaec3c8e-tmp-dir\") pod \"dns-default-ptnmn\" (UID: \"3532d5a1-91d6-426c-8e98-a6afaaec3c8e\") " pod="openshift-dns/dns-default-ptnmn" Jul 06 19:00:13.301710 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.301694 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3532d5a1-91d6-426c-8e98-a6afaaec3c8e-tmp-dir\") pod \"dns-default-ptnmn\" (UID: \"3532d5a1-91d6-426c-8e98-a6afaaec3c8e\") " pod="openshift-dns/dns-default-ptnmn" Jul 06 19:00:13.301930 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.301912 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3532d5a1-91d6-426c-8e98-a6afaaec3c8e-config-volume\") pod \"dns-default-ptnmn\" (UID: \"3532d5a1-91d6-426c-8e98-a6afaaec3c8e\") " pod="openshift-dns/dns-default-ptnmn" Jul 06 19:00:13.304995 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.304974 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3532d5a1-91d6-426c-8e98-a6afaaec3c8e-metrics-tls\") pod \"dns-default-ptnmn\" (UID: \"3532d5a1-91d6-426c-8e98-a6afaaec3c8e\") " pod="openshift-dns/dns-default-ptnmn" Jul 06 19:00:13.305082 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.305022 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7605339-9208-4293-b7f5-0573dd8d65de-cert\") pod \"ingress-canary-8sq64\" (UID: \"b7605339-9208-4293-b7f5-0573dd8d65de\") " pod="openshift-ingress-canary/ingress-canary-8sq64" Jul 06 19:00:13.333390 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.333365 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpngw\" (UniqueName: \"kubernetes.io/projected/3532d5a1-91d6-426c-8e98-a6afaaec3c8e-kube-api-access-rpngw\") pod \"dns-default-ptnmn\" (UID: \"3532d5a1-91d6-426c-8e98-a6afaaec3c8e\") " pod="openshift-dns/dns-default-ptnmn" Jul 06 19:00:13.333901 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.333880 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5nt4\" (UniqueName: \"kubernetes.io/projected/b7605339-9208-4293-b7f5-0573dd8d65de-kube-api-access-q5nt4\") pod \"ingress-canary-8sq64\" (UID: \"b7605339-9208-4293-b7f5-0573dd8d65de\") " pod="openshift-ingress-canary/ingress-canary-8sq64" Jul 06 19:00:13.401878 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.401848 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dd8fd2e7-6baa-43ec-85c7-457e042c3834-crio-socket\") pod \"insights-runtime-extractor-q725j\" (UID: \"dd8fd2e7-6baa-43ec-85c7-457e042c3834\") " pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.401878 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.401883 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd8fd2e7-6baa-43ec-85c7-457e042c3834-data-volume\") pod \"insights-runtime-extractor-q725j\" (UID: \"dd8fd2e7-6baa-43ec-85c7-457e042c3834\") " pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.402128 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.401907 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtlqr\" (UniqueName: \"kubernetes.io/projected/dd8fd2e7-6baa-43ec-85c7-457e042c3834-kube-api-access-vtlqr\") pod \"insights-runtime-extractor-q725j\" (UID: \"dd8fd2e7-6baa-43ec-85c7-457e042c3834\") " pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.402128 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.401998 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd8fd2e7-6baa-43ec-85c7-457e042c3834-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q725j\" (UID: \"dd8fd2e7-6baa-43ec-85c7-457e042c3834\") " pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.402128 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.402036 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd8fd2e7-6baa-43ec-85c7-457e042c3834-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q725j\" (UID: \"dd8fd2e7-6baa-43ec-85c7-457e042c3834\") " pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.402128 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.402106 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dd8fd2e7-6baa-43ec-85c7-457e042c3834-crio-socket\") pod \"insights-runtime-extractor-q725j\" (UID: \"dd8fd2e7-6baa-43ec-85c7-457e042c3834\") " pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.402276 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.402240 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd8fd2e7-6baa-43ec-85c7-457e042c3834-data-volume\") pod \"insights-runtime-extractor-q725j\" (UID: \"dd8fd2e7-6baa-43ec-85c7-457e042c3834\") " pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.402561 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.402545 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd8fd2e7-6baa-43ec-85c7-457e042c3834-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q725j\" (UID: \"dd8fd2e7-6baa-43ec-85c7-457e042c3834\") " pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.404256 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.404241 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd8fd2e7-6baa-43ec-85c7-457e042c3834-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q725j\" (UID: \"dd8fd2e7-6baa-43ec-85c7-457e042c3834\") " pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.420005 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.419982 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtlqr\" (UniqueName: \"kubernetes.io/projected/dd8fd2e7-6baa-43ec-85c7-457e042c3834-kube-api-access-vtlqr\") pod \"insights-runtime-extractor-q725j\" (UID: \"dd8fd2e7-6baa-43ec-85c7-457e042c3834\") " pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.420569 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.420542 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:13.434361 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.434338 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8sq64" Jul 06 19:00:13.446635 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.446617 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ptnmn" Jul 06 19:00:13.568710 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.568680 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q725j" Jul 06 19:00:13.674425 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.674393 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:13.734153 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.734118 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8sq64"] Jul 06 19:00:13.737835 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.737803 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ptnmn"] Jul 06 19:00:13.737961 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:00:13.737916 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7605339_9208_4293_b7f5_0573dd8d65de.slice/crio-259c035899dab9ba9d735db7322b244329b861d10c8a46356e8ec7ad8ac960c0 WatchSource:0}: Error finding container 259c035899dab9ba9d735db7322b244329b861d10c8a46356e8ec7ad8ac960c0: Status 404 returned error can't find the container with id 259c035899dab9ba9d735db7322b244329b861d10c8a46356e8ec7ad8ac960c0 Jul 06 19:00:13.740612 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:00:13.740589 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3532d5a1_91d6_426c_8e98_a6afaaec3c8e.slice/crio-286818308a2b72202bb66f9cea0964305b07fb09611d950bc69dbed8d4938a26 WatchSource:0}: Error finding container 286818308a2b72202bb66f9cea0964305b07fb09611d950bc69dbed8d4938a26: Status 404 returned error can't find the container with id 286818308a2b72202bb66f9cea0964305b07fb09611d950bc69dbed8d4938a26 Jul 06 19:00:13.763376 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:00:13.763351 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd8fd2e7_6baa_43ec_85c7_457e042c3834.slice/crio-08c82e8c3c533a30f0d291d866865c5cfb53ecc4d405c31490f008f7999b8083 WatchSource:0}: Error finding container 08c82e8c3c533a30f0d291d866865c5cfb53ecc4d405c31490f008f7999b8083: Status 404 returned error can't find the container with id 08c82e8c3c533a30f0d291d866865c5cfb53ecc4d405c31490f008f7999b8083 Jul 06 19:00:13.799144 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.799112 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q725j"] Jul 06 19:00:13.953823 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:13.953737 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:14.106852 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.106819 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-xd58h"] Jul 06 19:00:14.140194 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.140161 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bkrg4"] Jul 06 19:00:14.140341 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.140280 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-xd58h" Jul 06 19:00:14.144406 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.144379 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 19:00:14.144557 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.144510 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-wd6dn\"" Jul 06 19:00:14.144831 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.144813 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 19:00:14.152326 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.152304 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-xd58h"] Jul 06 19:00:14.152429 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.152341 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bkrg4"] Jul 06 19:00:14.152429 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.152418 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bkrg4" Jul 06 19:00:14.155388 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.155370 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 19:00:14.155522 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.155422 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-bccpm\"" Jul 06 19:00:14.160479 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.157891 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:14.190261 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.190230 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 19:00:14.190504 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.190228 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 19:00:14.193461 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.193265 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:00:14.193461 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.193277 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zn9xt\"" Jul 06 19:00:14.193461 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.193381 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:00:14.193461 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.193403 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:00:14.207565 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.207520 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zd9z\" (UniqueName: \"kubernetes.io/projected/bf18ee49-d386-429b-b848-ed10ff8f5280-kube-api-access-7zd9z\") pod \"downloads-685bfb5f64-xd58h\" (UID: \"bf18ee49-d386-429b-b848-ed10ff8f5280\") " pod="openshift-console/downloads-685bfb5f64-xd58h" Jul 06 19:00:14.207565 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.207565 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/76656ba1-38dc-4c56-8667-2624920898ed-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-bkrg4\" (UID: \"76656ba1-38dc-4c56-8667-2624920898ed\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bkrg4" Jul 06 19:00:14.290494 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.290458 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:14.308600 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.308568 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/76656ba1-38dc-4c56-8667-2624920898ed-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-bkrg4\" (UID: \"76656ba1-38dc-4c56-8667-2624920898ed\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bkrg4" Jul 06 19:00:14.308700 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.308667 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zd9z\" (UniqueName: \"kubernetes.io/projected/bf18ee49-d386-429b-b848-ed10ff8f5280-kube-api-access-7zd9z\") pod \"downloads-685bfb5f64-xd58h\" (UID: \"bf18ee49-d386-429b-b848-ed10ff8f5280\") " pod="openshift-console/downloads-685bfb5f64-xd58h" Jul 06 19:00:14.308765 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:14.308750 2557 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 06 19:00:14.308832 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:14.308822 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76656ba1-38dc-4c56-8667-2624920898ed-tls-certificates podName:76656ba1-38dc-4c56-8667-2624920898ed nodeName:}" failed. No retries permitted until 2026-07-06 19:00:14.808804201 +0000 UTC m=+55.169967986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/76656ba1-38dc-4c56-8667-2624920898ed-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-bkrg4" (UID: "76656ba1-38dc-4c56-8667-2624920898ed") : secret "prometheus-operator-admission-webhook-tls" not found Jul 06 19:00:14.325749 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.325719 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zd9z\" (UniqueName: \"kubernetes.io/projected/bf18ee49-d386-429b-b848-ed10ff8f5280-kube-api-access-7zd9z\") pod \"downloads-685bfb5f64-xd58h\" (UID: \"bf18ee49-d386-429b-b848-ed10ff8f5280\") " pod="openshift-console/downloads-685bfb5f64-xd58h" Jul 06 19:00:14.449673 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.449638 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-xd58h" Jul 06 19:00:14.510080 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.509320 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q725j" event={"ID":"dd8fd2e7-6baa-43ec-85c7-457e042c3834","Type":"ContainerStarted","Data":"08c82e8c3c533a30f0d291d866865c5cfb53ecc4d405c31490f008f7999b8083"} Jul 06 19:00:14.512005 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.511955 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ptnmn" event={"ID":"3532d5a1-91d6-426c-8e98-a6afaaec3c8e","Type":"ContainerStarted","Data":"286818308a2b72202bb66f9cea0964305b07fb09611d950bc69dbed8d4938a26"} Jul 06 19:00:14.513417 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.513362 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8sq64" event={"ID":"b7605339-9208-4293-b7f5-0573dd8d65de","Type":"ContainerStarted","Data":"259c035899dab9ba9d735db7322b244329b861d10c8a46356e8ec7ad8ac960c0"} Jul 06 19:00:14.535285 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.535021 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:14.664200 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.663352 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-xd58h"] Jul 06 19:00:14.667800 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:00:14.667766 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf18ee49_d386_429b_b848_ed10ff8f5280.slice/crio-86fc97520f945663ddfbbfddede9b089540d999842ab3afd5b299d3e520df80b WatchSource:0}: Error finding container 86fc97520f945663ddfbbfddede9b089540d999842ab3afd5b299d3e520df80b: Status 404 returned error can't find the container with id 86fc97520f945663ddfbbfddede9b089540d999842ab3afd5b299d3e520df80b Jul 06 19:00:14.781470 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.781392 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:14.813745 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.813694 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/76656ba1-38dc-4c56-8667-2624920898ed-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-bkrg4\" (UID: \"76656ba1-38dc-4c56-8667-2624920898ed\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bkrg4" Jul 06 19:00:14.817729 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.817631 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/76656ba1-38dc-4c56-8667-2624920898ed-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-bkrg4\" (UID: \"76656ba1-38dc-4c56-8667-2624920898ed\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bkrg4" Jul 06 19:00:14.880566 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:14.880321 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:15.063718 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:15.063323 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bkrg4" Jul 06 19:00:15.252418 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:15.252302 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bkrg4"] Jul 06 19:00:15.263454 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:00:15.263401 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76656ba1_38dc_4c56_8667_2624920898ed.slice/crio-462556db08bd84fd99ee10d09bbc952ae32d82b8d80a7e89a64a76638a120169 WatchSource:0}: Error finding container 462556db08bd84fd99ee10d09bbc952ae32d82b8d80a7e89a64a76638a120169: Status 404 returned error can't find the container with id 462556db08bd84fd99ee10d09bbc952ae32d82b8d80a7e89a64a76638a120169 Jul 06 19:00:15.403905 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:15.403732 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:15.506088 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:15.506033 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:15.519049 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:15.519000 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-xd58h" event={"ID":"bf18ee49-d386-429b-b848-ed10ff8f5280","Type":"ContainerStarted","Data":"86fc97520f945663ddfbbfddede9b089540d999842ab3afd5b299d3e520df80b"} Jul 06 19:00:15.520823 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:15.520795 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bkrg4" event={"ID":"76656ba1-38dc-4c56-8667-2624920898ed","Type":"ContainerStarted","Data":"462556db08bd84fd99ee10d09bbc952ae32d82b8d80a7e89a64a76638a120169"} Jul 06 19:00:15.606365 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:15.606204 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:15.708702 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:15.708614 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:15.891540 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:15.891481 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:16.086892 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:16.086845 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:16.328218 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:16.328148 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:16.527835 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:16.527799 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ptnmn" event={"ID":"3532d5a1-91d6-426c-8e98-a6afaaec3c8e","Type":"ContainerStarted","Data":"0c73c5b1b00bd6c348db1fda87ddf49f0100a9025172567250386dd89bba22f9"} Jul 06 19:00:16.990512 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:16.990418 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:17.186191 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:17.186149 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:17.439251 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:17.439167 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:20.545731 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:20.545695 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q725j" event={"ID":"dd8fd2e7-6baa-43ec-85c7-457e042c3834","Type":"ContainerStarted","Data":"4e66bd25329aa7047fa098acab8f64a4552e8f60186be1f2ec57115dbf4eb19d"} Jul 06 19:00:20.547824 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:20.547621 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bkrg4" event={"ID":"76656ba1-38dc-4c56-8667-2624920898ed","Type":"ContainerStarted","Data":"286eda567edfbc319036341098c6802f9d84bacd24803490f6ae13a3f622254d"} Jul 06 19:00:20.548165 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:20.548130 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bkrg4" Jul 06 19:00:20.550580 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:20.550555 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ptnmn" event={"ID":"3532d5a1-91d6-426c-8e98-a6afaaec3c8e","Type":"ContainerStarted","Data":"3906f1c724826b6e7fa884e64c9f3f3689b2b20496ec8ef4a9c1114ef1914023"} Jul 06 19:00:20.550722 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:20.550703 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ptnmn" Jul 06 19:00:20.553013 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:20.552987 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8sq64" event={"ID":"b7605339-9208-4293-b7f5-0573dd8d65de","Type":"ContainerStarted","Data":"b546f6a19adc65638e2bbac3e753859c655dbc8333778f7c4bb4559f9cb56d3c"} Jul 06 19:00:20.553925 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:20.553905 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bkrg4" Jul 06 19:00:20.563639 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:20.563591 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bkrg4" podStartSLOduration=2.019532196 podStartE2EDuration="6.563576439s" totalImagesPullingTime="4.544044243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:20.56281385 +0000 UTC m=+60.923977642" watchObservedRunningTime="2026-07-06 19:00:20.563576439 +0000 UTC m=+60.924740232" Jul 06 19:00:20.581889 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:20.581835 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ptnmn" podStartSLOduration=5.0787436360000004 podStartE2EDuration="7.581812854s" totalImagesPullingTime="2.503069218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:20.581488608 +0000 UTC m=+60.942652402" watchObservedRunningTime="2026-07-06 19:00:20.581812854 +0000 UTC m=+60.942976647" Jul 06 19:00:20.600971 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:20.600911 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8sq64" podStartSLOduration=1.520248544 podStartE2EDuration="7.600888564s" totalImagesPullingTime="6.08064002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:20.600746652 +0000 UTC m=+60.961910444" watchObservedRunningTime="2026-07-06 19:00:20.600888564 +0000 UTC m=+60.962052354" Jul 06 19:00:21.140164 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.140138 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t"] Jul 06 19:00:21.161716 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.161678 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t"] Jul 06 19:00:21.161865 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.161805 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" Jul 06 19:00:21.164774 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.164743 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-cz4gf\"" Jul 06 19:00:21.166121 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.166100 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 19:00:21.166354 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.166334 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 19:00:21.166519 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.166501 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 19:00:21.166751 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.166726 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 19:00:21.166855 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.166791 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 19:00:21.267939 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.267898 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-fcj9t\" (UID: \"2d6e5eba-0788-469a-858a-fdd5cf5be8a8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" Jul 06 19:00:21.268143 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.267967 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-fcj9t\" (UID: \"2d6e5eba-0788-469a-858a-fdd5cf5be8a8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" Jul 06 19:00:21.268143 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.268019 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8qzr\" (UniqueName: \"kubernetes.io/projected/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-kube-api-access-w8qzr\") pod \"prometheus-operator-567f7b8b7c-fcj9t\" (UID: \"2d6e5eba-0788-469a-858a-fdd5cf5be8a8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" Jul 06 19:00:21.268143 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.268107 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-fcj9t\" (UID: \"2d6e5eba-0788-469a-858a-fdd5cf5be8a8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" Jul 06 19:00:21.369484 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.369384 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8qzr\" (UniqueName: \"kubernetes.io/projected/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-kube-api-access-w8qzr\") pod \"prometheus-operator-567f7b8b7c-fcj9t\" (UID: \"2d6e5eba-0788-469a-858a-fdd5cf5be8a8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" Jul 06 19:00:21.369484 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.369456 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-fcj9t\" (UID: \"2d6e5eba-0788-469a-858a-fdd5cf5be8a8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" Jul 06 19:00:21.369713 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.369494 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-fcj9t\" (UID: \"2d6e5eba-0788-469a-858a-fdd5cf5be8a8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" Jul 06 19:00:21.369713 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.369516 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-fcj9t\" (UID: \"2d6e5eba-0788-469a-858a-fdd5cf5be8a8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" Jul 06 19:00:21.369713 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:21.369600 2557 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 06 19:00:21.369713 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:21.369649 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-prometheus-operator-tls podName:2d6e5eba-0788-469a-858a-fdd5cf5be8a8 nodeName:}" failed. No retries permitted until 2026-07-06 19:00:21.86963499 +0000 UTC m=+62.230798760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-fcj9t" (UID: "2d6e5eba-0788-469a-858a-fdd5cf5be8a8") : secret "prometheus-operator-tls" not found Jul 06 19:00:21.370288 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.370263 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-fcj9t\" (UID: \"2d6e5eba-0788-469a-858a-fdd5cf5be8a8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" Jul 06 19:00:21.371749 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.371721 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-fcj9t\" (UID: \"2d6e5eba-0788-469a-858a-fdd5cf5be8a8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" Jul 06 19:00:21.379310 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.379286 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8qzr\" (UniqueName: \"kubernetes.io/projected/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-kube-api-access-w8qzr\") pod \"prometheus-operator-567f7b8b7c-fcj9t\" (UID: \"2d6e5eba-0788-469a-858a-fdd5cf5be8a8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" Jul 06 19:00:21.458403 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.458375 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6w5n" Jul 06 19:00:21.504230 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.504124 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:21.560385 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.560347 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q725j" event={"ID":"dd8fd2e7-6baa-43ec-85c7-457e042c3834","Type":"ContainerStarted","Data":"e346b2dd6f3a7a04f413d8d6ba5d85ff45944c9f8268fd8c085aff14873fbc6d"} Jul 06 19:00:21.604766 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.604718 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:21.676604 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.676517 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:21.874283 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.874242 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-fcj9t\" (UID: \"2d6e5eba-0788-469a-858a-fdd5cf5be8a8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" Jul 06 19:00:21.874505 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:21.874431 2557 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 06 19:00:21.874583 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:21.874536 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-prometheus-operator-tls podName:2d6e5eba-0788-469a-858a-fdd5cf5be8a8 nodeName:}" failed. No retries permitted until 2026-07-06 19:00:22.874513573 +0000 UTC m=+63.235677356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-fcj9t" (UID: "2d6e5eba-0788-469a-858a-fdd5cf5be8a8") : secret "prometheus-operator-tls" not found Jul 06 19:00:21.985286 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:21.985170 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:22.882391 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:22.882348 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-fcj9t\" (UID: \"2d6e5eba-0788-469a-858a-fdd5cf5be8a8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" Jul 06 19:00:22.885291 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:22.885209 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d6e5eba-0788-469a-858a-fdd5cf5be8a8-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-fcj9t\" (UID: \"2d6e5eba-0788-469a-858a-fdd5cf5be8a8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" Jul 06 19:00:22.973404 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:22.973366 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" Jul 06 19:00:23.850576 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:23.850544 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t"] Jul 06 19:00:23.852836 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:00:23.852807 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d6e5eba_0788_469a_858a_fdd5cf5be8a8.slice/crio-c2c1c88f995ac7315cd21238347eaedc232a15d1d2dc447c4892f057637ad7be WatchSource:0}: Error finding container c2c1c88f995ac7315cd21238347eaedc232a15d1d2dc447c4892f057637ad7be: Status 404 returned error can't find the container with id c2c1c88f995ac7315cd21238347eaedc232a15d1d2dc447c4892f057637ad7be Jul 06 19:00:23.922618 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:23.922580 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:24.022885 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.022843 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:24.100882 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.100844 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-696dc88d99-mqwl4"] Jul 06 19:00:24.103107 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.103084 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.107589 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.107319 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 19:00:24.107589 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.107361 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 19:00:24.107589 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.107386 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 19:00:24.107589 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.107454 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 19:00:24.107589 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.107338 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vzl4z\"" Jul 06 19:00:24.107589 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.107323 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 19:00:24.115038 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.114993 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-696dc88d99-mqwl4"] Jul 06 19:00:24.192278 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.192248 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-service-ca\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.192487 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.192288 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-serving-cert\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.192487 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.192311 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-config\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.192487 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.192375 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-oauth-config\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.192487 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.192402 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-oauth-serving-cert\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.192487 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.192420 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8bkl\" (UniqueName: \"kubernetes.io/projected/5c125efd-4ffd-4d90-ac8a-ea2133b24143-kube-api-access-f8bkl\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.293527 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.293407 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-service-ca\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.293527 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.293486 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-serving-cert\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.293527 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.293511 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-config\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.293807 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.293558 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-oauth-config\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.293807 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.293586 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-oauth-serving-cert\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.293807 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.293611 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8bkl\" (UniqueName: \"kubernetes.io/projected/5c125efd-4ffd-4d90-ac8a-ea2133b24143-kube-api-access-f8bkl\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.294301 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.294252 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-service-ca\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.294426 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.294331 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-config\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.294525 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.294513 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-oauth-serving-cert\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.296385 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.296360 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-oauth-config\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.296510 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.296412 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-serving-cert\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.304398 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.304377 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8bkl\" (UniqueName: \"kubernetes.io/projected/5c125efd-4ffd-4d90-ac8a-ea2133b24143-kube-api-access-f8bkl\") pod \"console-696dc88d99-mqwl4\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.414670 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.414631 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:24.534297 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.534213 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:24.560232 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.560154 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-696dc88d99-mqwl4"] Jul 06 19:00:24.563563 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:00:24.563532 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c125efd_4ffd_4d90_ac8a_ea2133b24143.slice/crio-0b529cf7879dc66582ef611e888e6ed61a65bbeb82c216beb9eeaab927238358 WatchSource:0}: Error finding container 0b529cf7879dc66582ef611e888e6ed61a65bbeb82c216beb9eeaab927238358: Status 404 returned error can't find the container with id 0b529cf7879dc66582ef611e888e6ed61a65bbeb82c216beb9eeaab927238358 Jul 06 19:00:24.572224 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.572189 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696dc88d99-mqwl4" event={"ID":"5c125efd-4ffd-4d90-ac8a-ea2133b24143","Type":"ContainerStarted","Data":"0b529cf7879dc66582ef611e888e6ed61a65bbeb82c216beb9eeaab927238358"} Jul 06 19:00:24.575349 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.575310 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q725j" event={"ID":"dd8fd2e7-6baa-43ec-85c7-457e042c3834","Type":"ContainerStarted","Data":"9dafdc176759e063697d3bf90e44bae2a9478bef3d00945e5c8101fc456bae4d"} Jul 06 19:00:24.576910 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.576873 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" event={"ID":"2d6e5eba-0788-469a-858a-fdd5cf5be8a8","Type":"ContainerStarted","Data":"c2c1c88f995ac7315cd21238347eaedc232a15d1d2dc447c4892f057637ad7be"} Jul 06 19:00:24.600541 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.600484 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-q725j" podStartSLOduration=1.809619499 podStartE2EDuration="11.600463256s" totalImagesPullingTime="9.790843757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:00:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:24.599208355 +0000 UTC m=+64.960372148" watchObservedRunningTime="2026-07-06 19:00:24.600463256 +0000 UTC m=+64.961627049" Jul 06 19:00:24.655014 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.654978 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:24.741746 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:24.741710 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:25.587518 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:25.587480 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" event={"ID":"2d6e5eba-0788-469a-858a-fdd5cf5be8a8","Type":"ContainerStarted","Data":"d4280217ed4ed3c7a66c3da2aa1e161798e97aa3e9631b87c34d2fd621adbdf6"} Jul 06 19:00:25.588000 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:25.587579 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" event={"ID":"2d6e5eba-0788-469a-858a-fdd5cf5be8a8","Type":"ContainerStarted","Data":"f4c3b7daca9ffb8199cd55c72dd25d58340205d912542a9cdf7daadcb9f1bf79"} Jul 06 19:00:25.610167 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:25.610117 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-fcj9t" podStartSLOduration=3.064420973 podStartE2EDuration="4.610102482s" totalImagesPullingTime="1.545681509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:25.60880555 +0000 UTC m=+65.969969341" watchObservedRunningTime="2026-07-06 19:00:25.610102482 +0000 UTC m=+65.971266268" Jul 06 19:00:25.809235 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:25.809173 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs\") pod \"network-metrics-daemon-2hf6t\" (UID: \"cd62ad65-3e90-46f3-bdb6-67b9cb3bd158\") " pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 19:00:25.813389 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:25.813361 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:00:25.822648 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:25.822621 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd62ad65-3e90-46f3-bdb6-67b9cb3bd158-metrics-certs\") pod \"network-metrics-daemon-2hf6t\" (UID: \"cd62ad65-3e90-46f3-bdb6-67b9cb3bd158\") " pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 19:00:25.830277 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:25.830202 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:25.910609 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:25.910570 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv88d\" (UniqueName: \"kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d\") pod \"network-check-target-69gb8\" (UID: \"794f47cc-dce2-43e8-b571-d431f4b6cec0\") " pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 19:00:25.910609 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:25.910622 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret\") pod \"global-pull-secret-syncer-k9pnm\" (UID: \"04e067b8-9d48-42c6-8cac-0254ea3a4328\") " pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 19:00:25.914702 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:25.914669 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:00:25.914841 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:25.914747 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:00:25.923519 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:25.923410 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:00:25.923655 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:25.923577 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e067b8-9d48-42c6-8cac-0254ea3a4328-original-pull-secret\") pod \"global-pull-secret-syncer-k9pnm\" (UID: \"04e067b8-9d48-42c6-8cac-0254ea3a4328\") " pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 19:00:25.933482 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:25.933456 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv88d\" (UniqueName: \"kubernetes.io/projected/794f47cc-dce2-43e8-b571-d431f4b6cec0-kube-api-access-vv88d\") pod \"network-check-target-69gb8\" (UID: \"794f47cc-dce2-43e8-b571-d431f4b6cec0\") " pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 19:00:26.102877 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:26.102795 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6kw7z\"" Jul 06 19:00:26.110575 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:26.110504 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hf6t" Jul 06 19:00:26.203611 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:26.203315 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zn9xt\"" Jul 06 19:00:26.204822 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:26.204796 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9pnm" Jul 06 19:00:26.211212 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:26.211161 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 19:00:26.272395 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:26.272184 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2hf6t"] Jul 06 19:00:26.328604 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:26.328570 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:26.365734 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:26.365645 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k9pnm"] Jul 06 19:00:26.370110 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:00:26.370076 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04e067b8_9d48_42c6_8cac_0254ea3a4328.slice/crio-748ad031b72501f504c1198c6cc81ddca27e56f34bc39ca2482e2b84c02f4f53 WatchSource:0}: Error finding container 748ad031b72501f504c1198c6cc81ddca27e56f34bc39ca2482e2b84c02f4f53: Status 404 returned error can't find the container with id 748ad031b72501f504c1198c6cc81ddca27e56f34bc39ca2482e2b84c02f4f53 Jul 06 19:00:26.400921 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:26.400827 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-69gb8"] Jul 06 19:00:26.404669 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:00:26.404195 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod794f47cc_dce2_43e8_b571_d431f4b6cec0.slice/crio-f43b832a33b1b72aff35af199e5ff59e6da2769ace6efeb9d836de1fab4196cb WatchSource:0}: Error finding container f43b832a33b1b72aff35af199e5ff59e6da2769ace6efeb9d836de1fab4196cb: Status 404 returned error can't find the container with id f43b832a33b1b72aff35af199e5ff59e6da2769ace6efeb9d836de1fab4196cb Jul 06 19:00:26.438326 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:26.438275 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:26.592629 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:26.592589 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2hf6t" event={"ID":"cd62ad65-3e90-46f3-bdb6-67b9cb3bd158","Type":"ContainerStarted","Data":"45891097e6a2428daabba634ccb99939e51b1419af10014a6db45f8ed8b7a8c8"} Jul 06 19:00:26.593972 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:26.593945 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k9pnm" event={"ID":"04e067b8-9d48-42c6-8cac-0254ea3a4328","Type":"ContainerStarted","Data":"748ad031b72501f504c1198c6cc81ddca27e56f34bc39ca2482e2b84c02f4f53"} Jul 06 19:00:26.595350 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:26.595311 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-69gb8" event={"ID":"794f47cc-dce2-43e8-b571-d431f4b6cec0","Type":"ContainerStarted","Data":"f43b832a33b1b72aff35af199e5ff59e6da2769ace6efeb9d836de1fab4196cb"} Jul 06 19:00:27.414412 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.414360 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:27.517765 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.517591 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:27.564559 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.563637 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ptnmn" Jul 06 19:00:27.564717 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.564699 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7"] Jul 06 19:00:27.591696 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.591036 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" Jul 06 19:00:27.592805 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.592714 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7"] Jul 06 19:00:27.595662 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.595198 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 19:00:27.595662 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.595308 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 19:00:27.595662 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.595553 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-kt54p\"" Jul 06 19:00:27.598852 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.598285 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vnsbs"] Jul 06 19:00:27.619575 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.619513 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:27.621171 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.621139 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd"] Jul 06 19:00:27.622085 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.621394 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.625989 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.624934 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77d08ee5-6984-47b3-8e6c-9645bc5afe52-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-9j6j7\" (UID: \"77d08ee5-6984-47b3-8e6c-9645bc5afe52\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" Jul 06 19:00:27.625989 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.624973 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77d08ee5-6984-47b3-8e6c-9645bc5afe52-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-9j6j7\" (UID: \"77d08ee5-6984-47b3-8e6c-9645bc5afe52\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" Jul 06 19:00:27.625989 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.625030 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mh6h\" (UniqueName: \"kubernetes.io/projected/77d08ee5-6984-47b3-8e6c-9645bc5afe52-kube-api-access-7mh6h\") pod \"openshift-state-metrics-54cf96dfcf-9j6j7\" (UID: \"77d08ee5-6984-47b3-8e6c-9645bc5afe52\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" Jul 06 19:00:27.625989 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.625070 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77d08ee5-6984-47b3-8e6c-9645bc5afe52-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-9j6j7\" (UID: \"77d08ee5-6984-47b3-8e6c-9645bc5afe52\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" Jul 06 19:00:27.626247 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.626043 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 19:00:27.626247 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.626194 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 19:00:27.626985 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.626906 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 19:00:27.628199 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.628018 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cr5b7\"" Jul 06 19:00:27.645271 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.645243 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd"] Jul 06 19:00:27.645428 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.645416 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.648345 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.648323 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 19:00:27.648908 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.648872 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 19:00:27.649532 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.649094 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fl6cn\"" Jul 06 19:00:27.651464 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.650937 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 19:00:27.725847 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.725762 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5e3dcc7d-c527-43c9-8d48-35fccf277853-node-exporter-wtmp\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.725847 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.725816 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5e3dcc7d-c527-43c9-8d48-35fccf277853-node-exporter-accelerators-collector-config\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.726022 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.725869 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96w8g\" (UniqueName: \"kubernetes.io/projected/5e3dcc7d-c527-43c9-8d48-35fccf277853-kube-api-access-96w8g\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.726022 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.725892 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.726022 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.725949 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77d08ee5-6984-47b3-8e6c-9645bc5afe52-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-9j6j7\" (UID: \"77d08ee5-6984-47b3-8e6c-9645bc5afe52\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" Jul 06 19:00:27.726022 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.725971 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77d08ee5-6984-47b3-8e6c-9645bc5afe52-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-9j6j7\" (UID: \"77d08ee5-6984-47b3-8e6c-9645bc5afe52\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" Jul 06 19:00:27.726022 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.726000 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5e3dcc7d-c527-43c9-8d48-35fccf277853-node-exporter-textfile\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.726266 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.726027 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5e3dcc7d-c527-43c9-8d48-35fccf277853-node-exporter-tls\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.726266 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.726059 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5drs\" (UniqueName: \"kubernetes.io/projected/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-kube-api-access-z5drs\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.726266 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.726089 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.726266 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.726136 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5e3dcc7d-c527-43c9-8d48-35fccf277853-root\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.726266 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.726167 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mh6h\" (UniqueName: \"kubernetes.io/projected/77d08ee5-6984-47b3-8e6c-9645bc5afe52-kube-api-access-7mh6h\") pod \"openshift-state-metrics-54cf96dfcf-9j6j7\" (UID: \"77d08ee5-6984-47b3-8e6c-9645bc5afe52\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" Jul 06 19:00:27.726266 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.726202 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e3dcc7d-c527-43c9-8d48-35fccf277853-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.726266 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.726236 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77d08ee5-6984-47b3-8e6c-9645bc5afe52-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-9j6j7\" (UID: \"77d08ee5-6984-47b3-8e6c-9645bc5afe52\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" Jul 06 19:00:27.726266 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.726265 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.726654 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.726301 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e3dcc7d-c527-43c9-8d48-35fccf277853-sys\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.726654 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.726331 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e3dcc7d-c527-43c9-8d48-35fccf277853-metrics-client-ca\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.726654 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.726358 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.726654 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.726387 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.727041 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.726990 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77d08ee5-6984-47b3-8e6c-9645bc5afe52-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-9j6j7\" (UID: \"77d08ee5-6984-47b3-8e6c-9645bc5afe52\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" Jul 06 19:00:27.729212 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.729189 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77d08ee5-6984-47b3-8e6c-9645bc5afe52-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-9j6j7\" (UID: \"77d08ee5-6984-47b3-8e6c-9645bc5afe52\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" Jul 06 19:00:27.729312 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.729294 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77d08ee5-6984-47b3-8e6c-9645bc5afe52-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-9j6j7\" (UID: \"77d08ee5-6984-47b3-8e6c-9645bc5afe52\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" Jul 06 19:00:27.743774 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.743741 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mh6h\" (UniqueName: \"kubernetes.io/projected/77d08ee5-6984-47b3-8e6c-9645bc5afe52-kube-api-access-7mh6h\") pod \"openshift-state-metrics-54cf96dfcf-9j6j7\" (UID: \"77d08ee5-6984-47b3-8e6c-9645bc5afe52\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" Jul 06 19:00:27.754883 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.754836 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:27.826882 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.826815 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.826882 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.826860 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5e3dcc7d-c527-43c9-8d48-35fccf277853-root\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.827134 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.826893 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e3dcc7d-c527-43c9-8d48-35fccf277853-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.827134 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.826925 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.827134 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.826959 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e3dcc7d-c527-43c9-8d48-35fccf277853-sys\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.827134 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.826992 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e3dcc7d-c527-43c9-8d48-35fccf277853-metrics-client-ca\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.827134 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.827022 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.827134 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.827050 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.827134 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.827092 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5e3dcc7d-c527-43c9-8d48-35fccf277853-node-exporter-wtmp\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.827134 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.827122 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5e3dcc7d-c527-43c9-8d48-35fccf277853-node-exporter-accelerators-collector-config\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.827548 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.827159 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96w8g\" (UniqueName: \"kubernetes.io/projected/5e3dcc7d-c527-43c9-8d48-35fccf277853-kube-api-access-96w8g\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.827548 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.827199 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.827548 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.827209 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e3dcc7d-c527-43c9-8d48-35fccf277853-sys\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.827548 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.827245 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5e3dcc7d-c527-43c9-8d48-35fccf277853-node-exporter-textfile\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.827548 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.827279 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5e3dcc7d-c527-43c9-8d48-35fccf277853-node-exporter-tls\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.827548 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.827311 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5e3dcc7d-c527-43c9-8d48-35fccf277853-root\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.827548 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.827326 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.828548 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.827324 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5drs\" (UniqueName: \"kubernetes.io/projected/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-kube-api-access-z5drs\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.828548 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:27.827952 2557 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 06 19:00:27.828548 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.827977 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5e3dcc7d-c527-43c9-8d48-35fccf277853-node-exporter-accelerators-collector-config\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.828548 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:27.828033 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-kube-state-metrics-tls podName:48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153 nodeName:}" failed. No retries permitted until 2026-07-06 19:00:28.328012426 +0000 UTC m=+68.689176219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-gsfgd" (UID: "48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153") : secret "kube-state-metrics-tls" not found Jul 06 19:00:27.828548 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.828087 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.828548 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.828233 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5e3dcc7d-c527-43c9-8d48-35fccf277853-node-exporter-wtmp\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.828892 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.828748 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.828892 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.828848 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5e3dcc7d-c527-43c9-8d48-35fccf277853-node-exporter-textfile\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.829247 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.829196 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e3dcc7d-c527-43c9-8d48-35fccf277853-metrics-client-ca\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.832047 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.831905 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.833084 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.832859 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e3dcc7d-c527-43c9-8d48-35fccf277853-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.833084 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.833041 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5e3dcc7d-c527-43c9-8d48-35fccf277853-node-exporter-tls\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.838375 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.838331 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5drs\" (UniqueName: \"kubernetes.io/projected/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-kube-api-access-z5drs\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:27.840071 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.840001 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96w8g\" (UniqueName: \"kubernetes.io/projected/5e3dcc7d-c527-43c9-8d48-35fccf277853-kube-api-access-96w8g\") pod \"node-exporter-vnsbs\" (UID: \"5e3dcc7d-c527-43c9-8d48-35fccf277853\") " pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.843416 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.843289 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:27.907388 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.907153 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" Jul 06 19:00:27.937969 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.937927 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vnsbs" Jul 06 19:00:27.974288 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:27.974239 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:28.071749 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:28.071585 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:28.240932 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:28.240892 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:28.332354 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:28.332266 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:28.336316 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:28.336281 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-gsfgd\" (UID: \"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:28.466060 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:28.466023 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:28.559027 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:28.558937 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" Jul 06 19:00:28.808322 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:28.808232 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:28.914817 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:28.914781 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:28.954570 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:28.954343 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:00:28.960603 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:28.960575 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:28.966795 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:28.966565 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-tls\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" type="*v1.Secret" Jul 06 19:00:28.966795 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:28.966586 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-kube-rbac-proxy-web\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" type="*v1.Secret" Jul 06 19:00:28.966795 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:28.966622 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-kube-rbac-proxy-metric\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" type="*v1.Secret" Jul 06 19:00:28.966795 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:28.966721 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-cluster-tls-config\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" type="*v1.Secret" Jul 06 19:00:28.966795 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:28.966761 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-dockercfg-fnh2m\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fnh2m\"" type="*v1.Secret" Jul 06 19:00:28.967190 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:28.967163 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"alertmanager-trusted-ca-bundle\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" type="*v1.ConfigMap" Jul 06 19:00:28.967873 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:28.967845 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-tls-assets-0\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" type="*v1.Secret" Jul 06 19:00:28.968204 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:28.968176 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-kube-rbac-proxy\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" type="*v1.Secret" Jul 06 19:00:28.969369 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:28.969337 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"alertmanager-main-0\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:28.969965 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:28.969688 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-web-config\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" type="*v1.Secret" Jul 06 19:00:28.969965 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:28.969765 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-generated\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" type="*v1.Secret" Jul 06 19:00:28.993700 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:28.993607 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:00:29.040347 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.039495 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlpx2\" (UniqueName: \"kubernetes.io/projected/1e373d67-33bb-4271-8d89-0f4f45c612e7-kube-api-access-xlpx2\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.040347 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.039547 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.040347 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.039603 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.040347 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.039637 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-config-volume\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.040347 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.039709 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1e373d67-33bb-4271-8d89-0f4f45c612e7-config-out\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.040347 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.039809 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1e373d67-33bb-4271-8d89-0f4f45c612e7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.040347 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.039906 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e373d67-33bb-4271-8d89-0f4f45c612e7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.040347 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.039942 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1e373d67-33bb-4271-8d89-0f4f45c612e7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.040347 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.039967 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-web-config\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.040347 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.040004 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.040347 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.040031 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.040347 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.040075 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e373d67-33bb-4271-8d89-0f4f45c612e7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.040347 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.040122 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.125870 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.125680 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:29.143517 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.140921 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-web-config\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.143517 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.140980 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.143517 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.141015 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.143517 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.141049 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e373d67-33bb-4271-8d89-0f4f45c612e7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.143517 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.141090 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.143517 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.141145 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlpx2\" (UniqueName: \"kubernetes.io/projected/1e373d67-33bb-4271-8d89-0f4f45c612e7-kube-api-access-xlpx2\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.143517 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.141175 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.143517 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.141231 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.143517 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.141264 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-config-volume\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.143517 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.141292 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1e373d67-33bb-4271-8d89-0f4f45c612e7-config-out\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.143517 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.141317 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1e373d67-33bb-4271-8d89-0f4f45c612e7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.143517 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.141349 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e373d67-33bb-4271-8d89-0f4f45c612e7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.143517 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.141381 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1e373d67-33bb-4271-8d89-0f4f45c612e7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.143517 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.142987 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1e373d67-33bb-4271-8d89-0f4f45c612e7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.144375 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.143847 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e373d67-33bb-4271-8d89-0f4f45c612e7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.145299 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.145275 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1e373d67-33bb-4271-8d89-0f4f45c612e7-config-out\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.159733 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.159703 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlpx2\" (UniqueName: \"kubernetes.io/projected/1e373d67-33bb-4271-8d89-0f4f45c612e7-kube-api-access-xlpx2\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.427694 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:00:29.427662 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e3dcc7d_c527_43c9_8d48_35fccf277853.slice/crio-7930eb07777d35f0141c3b1962b51c374e9940193613836412297d4b0a3324e9 WatchSource:0}: Error finding container 7930eb07777d35f0141c3b1962b51c374e9940193613836412297d4b0a3324e9: Status 404 returned error can't find the container with id 7930eb07777d35f0141c3b1962b51c374e9940193613836412297d4b0a3324e9 Jul 06 19:00:29.616849 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.616545 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2hf6t" event={"ID":"cd62ad65-3e90-46f3-bdb6-67b9cb3bd158","Type":"ContainerStarted","Data":"7685b98b1a297e7590e81f57579de37869928a9a27471f2dbef78b44555980b1"} Jul 06 19:00:29.617661 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.617598 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd"] Jul 06 19:00:29.618949 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:00:29.618831 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48fb1c5a_c99e_4dc4_88a3_f6b32aa8d153.slice/crio-7a57aa7bd7c882e760ff99a2651a3f4a3d717cadcc54c90b3584fec88f9c419f WatchSource:0}: Error finding container 7a57aa7bd7c882e760ff99a2651a3f4a3d717cadcc54c90b3584fec88f9c419f: Status 404 returned error can't find the container with id 7a57aa7bd7c882e760ff99a2651a3f4a3d717cadcc54c90b3584fec88f9c419f Jul 06 19:00:29.619424 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.619350 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vnsbs" event={"ID":"5e3dcc7d-c527-43c9-8d48-35fccf277853","Type":"ContainerStarted","Data":"7930eb07777d35f0141c3b1962b51c374e9940193613836412297d4b0a3324e9"} Jul 06 19:00:29.626005 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.625301 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696dc88d99-mqwl4" event={"ID":"5c125efd-4ffd-4d90-ac8a-ea2133b24143","Type":"ContainerStarted","Data":"65411cbf14e99e9f36e37dc88750eb0693455161fc1b2b2fca08f98a2fe89408"} Jul 06 19:00:29.639508 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.639477 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7"] Jul 06 19:00:29.643403 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:00:29.643372 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d08ee5_6984_47b3_8e6c_9645bc5afe52.slice/crio-8a9780e10511e4ac89c60bb86495850ab5b1594b68899c002ed7b01ae3b5be54 WatchSource:0}: Error finding container 8a9780e10511e4ac89c60bb86495850ab5b1594b68899c002ed7b01ae3b5be54: Status 404 returned error can't find the container with id 8a9780e10511e4ac89c60bb86495850ab5b1594b68899c002ed7b01ae3b5be54 Jul 06 19:00:29.680210 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.680142 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-696dc88d99-mqwl4" podStartSLOduration=0.806322444 podStartE2EDuration="5.680118485s" totalImagesPullingTime="4.873796041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:29.651143081 +0000 UTC m=+70.012306878" watchObservedRunningTime="2026-07-06 19:00:29.680118485 +0000 UTC m=+70.041282280" Jul 06 19:00:29.803091 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.803060 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 19:00:29.817087 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.816873 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.873214 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.872935 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 19:00:29.885795 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.885762 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-config-volume\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.970347 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.970255 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 19:00:29.975191 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.975160 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-web-config\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:29.978251 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:29.978146 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fnh2m\"" Jul 06 19:00:30.059077 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.059042 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 19:00:30.065207 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.065177 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1e373d67-33bb-4271-8d89-0f4f45c612e7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:30.105314 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.105281 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 19:00:30.115127 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.114922 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 19:00:30.116552 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.116527 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:30.122861 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.122815 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e373d67-33bb-4271-8d89-0f4f45c612e7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:30.142280 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:30.142247 2557 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-kube-rbac-proxy-metric: failed to sync secret cache: timed out waiting for the condition Jul 06 19:00:30.142465 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:30.142350 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy-metric podName:1e373d67-33bb-4271-8d89-0f4f45c612e7 nodeName:}" failed. No retries permitted until 2026-07-06 19:00:30.642327165 +0000 UTC m=+71.003490946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-kube-rbac-proxy-metric" (UniqueName: "kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy-metric") pod "alertmanager-main-0" (UID: "1e373d67-33bb-4271-8d89-0f4f45c612e7") : failed to sync secret cache: timed out waiting for the condition Jul 06 19:00:30.142465 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:30.142384 2557 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-cluster-tls-config: failed to sync secret cache: timed out waiting for the condition Jul 06 19:00:30.142465 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:30.142422 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-cluster-tls-config podName:1e373d67-33bb-4271-8d89-0f4f45c612e7 nodeName:}" failed. No retries permitted until 2026-07-06 19:00:30.642411171 +0000 UTC m=+71.003574949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-tls-config" (UniqueName: "kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-cluster-tls-config") pod "alertmanager-main-0" (UID: "1e373d67-33bb-4271-8d89-0f4f45c612e7") : failed to sync secret cache: timed out waiting for the condition Jul 06 19:00:30.142647 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:30.142494 2557 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-kube-rbac-proxy: failed to sync secret cache: timed out waiting for the condition Jul 06 19:00:30.142647 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:00:30.142542 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy podName:1e373d67-33bb-4271-8d89-0f4f45c612e7 nodeName:}" failed. No retries permitted until 2026-07-06 19:00:30.642517147 +0000 UTC m=+71.003680930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-kube-rbac-proxy" (UniqueName: "kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy") pod "alertmanager-main-0" (UID: "1e373d67-33bb-4271-8d89-0f4f45c612e7") : failed to sync secret cache: timed out waiting for the condition Jul 06 19:00:30.193070 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.193027 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 19:00:30.231649 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.231516 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 19:00:30.471192 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.471159 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 19:00:30.628610 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.628569 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:30.635017 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.634913 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" event={"ID":"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153","Type":"ContainerStarted","Data":"7a57aa7bd7c882e760ff99a2651a3f4a3d717cadcc54c90b3584fec88f9c419f"} Jul 06 19:00:30.639386 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.638939 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2hf6t" event={"ID":"cd62ad65-3e90-46f3-bdb6-67b9cb3bd158","Type":"ContainerStarted","Data":"95ed29225207f27a32c3e8296332970bb6a4d310082ca0a979d764a143d5b66d"} Jul 06 19:00:30.643036 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.642908 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" event={"ID":"77d08ee5-6984-47b3-8e6c-9645bc5afe52","Type":"ContainerStarted","Data":"e789b5a940a29e03303f854715c3e30e8a2115eaee64ca114e990eec18f47c58"} Jul 06 19:00:30.643036 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.642967 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" event={"ID":"77d08ee5-6984-47b3-8e6c-9645bc5afe52","Type":"ContainerStarted","Data":"734f2b21a7db77953673f5c4f6c81b1943951a23c2582e0f5d8340217d2abb88"} Jul 06 19:00:30.643036 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.642995 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" event={"ID":"77d08ee5-6984-47b3-8e6c-9645bc5afe52","Type":"ContainerStarted","Data":"8a9780e10511e4ac89c60bb86495850ab5b1594b68899c002ed7b01ae3b5be54"} Jul 06 19:00:30.654789 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.654755 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:30.655099 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.654821 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:30.655099 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.654893 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:30.658530 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.658470 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:30.659149 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.659093 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:30.659149 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.659134 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:30.674959 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.674894 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2hf6t" podStartSLOduration=67.541483337 podStartE2EDuration="1m10.674873725s" totalImagesPullingTime="3.133390388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:30.674565669 +0000 UTC m=+71.035729467" watchObservedRunningTime="2026-07-06 19:00:30.674873725 +0000 UTC m=+71.036037519" Jul 06 19:00:30.751171 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.751130 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:30.773873 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.773832 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:00:30.799985 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.799947 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp"] Jul 06 19:00:30.802995 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.802972 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.806176 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.806150 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-crsh4mf66lue5\"" Jul 06 19:00:30.806308 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.806154 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 19:00:30.806738 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.806673 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-9rmbr\"" Jul 06 19:00:30.806738 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.806698 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 19:00:30.806892 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.806755 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 19:00:30.806990 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.806969 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 19:00:30.806990 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.806984 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 19:00:30.828852 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.828822 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp"] Jul 06 19:00:30.857604 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.857569 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.857797 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.857619 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.857797 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.857650 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.857797 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.857718 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-thanos-querier-tls\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.857797 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.857746 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-grpc-tls\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.857989 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.857834 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.857989 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.857903 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sn4x\" (UniqueName: \"kubernetes.io/projected/5abc161e-79ec-4ba0-b063-084f93dfe838-kube-api-access-4sn4x\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.857989 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.857947 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5abc161e-79ec-4ba0-b063-084f93dfe838-metrics-client-ca\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.878318 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.878268 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:30.959392 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.959307 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.959392 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.959347 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.959392 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.959376 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.959706 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.959408 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-thanos-querier-tls\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.959706 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.959448 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-grpc-tls\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.959706 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.959498 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.959706 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.959546 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sn4x\" (UniqueName: \"kubernetes.io/projected/5abc161e-79ec-4ba0-b063-084f93dfe838-kube-api-access-4sn4x\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.959706 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.959578 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5abc161e-79ec-4ba0-b063-084f93dfe838-metrics-client-ca\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.960618 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.960329 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5abc161e-79ec-4ba0-b063-084f93dfe838-metrics-client-ca\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.962681 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.962656 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.962798 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.962656 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.963167 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.963109 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.963370 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.963262 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-thanos-querier-tls\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.963370 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.963321 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.963737 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.963720 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5abc161e-79ec-4ba0-b063-084f93dfe838-secret-grpc-tls\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:30.967113 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.967079 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:30.969649 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:30.969631 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sn4x\" (UniqueName: \"kubernetes.io/projected/5abc161e-79ec-4ba0-b063-084f93dfe838-kube-api-access-4sn4x\") pod \"thanos-querier-6b85fc5cc7-wpfrp\" (UID: \"5abc161e-79ec-4ba0-b063-084f93dfe838\") " pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:31.070171 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:31.070110 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:31.114420 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:31.114386 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:31.392780 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:31.392740 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:31.916476 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:31.916427 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:32.038043 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.037976 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:32.085796 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.085755 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-57b7fccbbf-5c2nh"] Jul 06 19:00:32.089388 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.089361 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.093768 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.093743 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 19:00:32.093768 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.093766 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 19:00:32.093977 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.093846 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qthhh\"" Jul 06 19:00:32.093977 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.093854 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 19:00:32.096835 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.096813 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-e2ncigno7v7qa\"" Jul 06 19:00:32.097599 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.097574 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 19:00:32.112772 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.112741 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-57b7fccbbf-5c2nh"] Jul 06 19:00:32.170190 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.170090 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/03b694d1-fffd-4360-867b-72db963a7a15-secret-metrics-server-client-certs\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.170190 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.170145 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/03b694d1-fffd-4360-867b-72db963a7a15-metrics-server-audit-profiles\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.170425 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.170232 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/03b694d1-fffd-4360-867b-72db963a7a15-audit-log\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.170425 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.170300 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03b694d1-fffd-4360-867b-72db963a7a15-client-ca-bundle\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.170425 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.170360 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/03b694d1-fffd-4360-867b-72db963a7a15-secret-metrics-server-tls\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.170425 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.170391 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fd2q\" (UniqueName: \"kubernetes.io/projected/03b694d1-fffd-4360-867b-72db963a7a15-kube-api-access-5fd2q\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.170643 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.170490 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03b694d1-fffd-4360-867b-72db963a7a15-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.271231 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.271185 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/03b694d1-fffd-4360-867b-72db963a7a15-audit-log\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.271427 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.271245 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03b694d1-fffd-4360-867b-72db963a7a15-client-ca-bundle\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.271427 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.271297 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/03b694d1-fffd-4360-867b-72db963a7a15-secret-metrics-server-tls\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.271427 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.271326 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fd2q\" (UniqueName: \"kubernetes.io/projected/03b694d1-fffd-4360-867b-72db963a7a15-kube-api-access-5fd2q\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.271603 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.271492 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03b694d1-fffd-4360-867b-72db963a7a15-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.271603 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.271567 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/03b694d1-fffd-4360-867b-72db963a7a15-secret-metrics-server-client-certs\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.271689 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.271601 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/03b694d1-fffd-4360-867b-72db963a7a15-metrics-server-audit-profiles\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.271689 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.271661 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/03b694d1-fffd-4360-867b-72db963a7a15-audit-log\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.272398 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.272356 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03b694d1-fffd-4360-867b-72db963a7a15-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.272767 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.272741 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/03b694d1-fffd-4360-867b-72db963a7a15-metrics-server-audit-profiles\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.274220 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.274195 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03b694d1-fffd-4360-867b-72db963a7a15-client-ca-bundle\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.274320 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.274198 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/03b694d1-fffd-4360-867b-72db963a7a15-secret-metrics-server-tls\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.274320 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.274314 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/03b694d1-fffd-4360-867b-72db963a7a15-secret-metrics-server-client-certs\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.291531 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.291504 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fd2q\" (UniqueName: \"kubernetes.io/projected/03b694d1-fffd-4360-867b-72db963a7a15-kube-api-access-5fd2q\") pod \"metrics-server-57b7fccbbf-5c2nh\" (UID: \"03b694d1-fffd-4360-867b-72db963a7a15\") " pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.368659 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.368620 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:32.393638 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.393603 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-d84pd"] Jul 06 19:00:32.397906 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.397885 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-d84pd" Jul 06 19:00:32.400834 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.400808 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:32.401707 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.401683 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-gvmx4\"" Jul 06 19:00:32.402013 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.401989 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 19:00:32.409501 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.409475 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-d84pd"] Jul 06 19:00:32.473857 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.473768 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz7cl\" (UniqueName: \"kubernetes.io/projected/5c8854d1-590f-4b85-8925-1cd1eb42dc13-kube-api-access-gz7cl\") pod \"monitoring-plugin-d5dcff7cf-d84pd\" (UID: \"5c8854d1-590f-4b85-8925-1cd1eb42dc13\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-d84pd" Jul 06 19:00:32.473857 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.473845 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5c8854d1-590f-4b85-8925-1cd1eb42dc13-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-d84pd\" (UID: \"5c8854d1-590f-4b85-8925-1cd1eb42dc13\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-d84pd" Jul 06 19:00:32.491634 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.491594 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:32.574843 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.574805 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz7cl\" (UniqueName: \"kubernetes.io/projected/5c8854d1-590f-4b85-8925-1cd1eb42dc13-kube-api-access-gz7cl\") pod \"monitoring-plugin-d5dcff7cf-d84pd\" (UID: \"5c8854d1-590f-4b85-8925-1cd1eb42dc13\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-d84pd" Jul 06 19:00:32.575033 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.574875 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5c8854d1-590f-4b85-8925-1cd1eb42dc13-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-d84pd\" (UID: \"5c8854d1-590f-4b85-8925-1cd1eb42dc13\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-d84pd" Jul 06 19:00:32.577676 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.577650 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5c8854d1-590f-4b85-8925-1cd1eb42dc13-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-d84pd\" (UID: \"5c8854d1-590f-4b85-8925-1cd1eb42dc13\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-d84pd" Jul 06 19:00:32.586221 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.586195 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz7cl\" (UniqueName: \"kubernetes.io/projected/5c8854d1-590f-4b85-8925-1cd1eb42dc13-kube-api-access-gz7cl\") pod \"monitoring-plugin-d5dcff7cf-d84pd\" (UID: \"5c8854d1-590f-4b85-8925-1cd1eb42dc13\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-d84pd" Jul 06 19:00:32.621776 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.621731 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:32.709784 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.709742 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-d84pd" Jul 06 19:00:32.937590 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.937548 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6455cb7df7-swbff"] Jul 06 19:00:32.941268 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.941240 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:32.945193 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.945173 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 06 19:00:32.945476 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.945449 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 06 19:00:32.953127 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.952374 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 06 19:00:32.953127 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.952568 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 06 19:00:32.953127 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.952783 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-k5jkv\"" Jul 06 19:00:32.953127 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.952987 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 06 19:00:32.969845 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.969816 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 06 19:00:32.978567 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:32.978536 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6455cb7df7-swbff"] Jul 06 19:00:33.079633 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.079592 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-secret-telemeter-client\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.079835 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.079652 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.079835 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.079684 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-telemeter-client-tls\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.079835 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.079782 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-serving-certs-ca-bundle\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.079835 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.079820 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c42s\" (UniqueName: \"kubernetes.io/projected/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-kube-api-access-6c42s\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.080057 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.079900 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.080057 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.079920 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-federate-client-tls\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.080057 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.079967 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-metrics-client-ca\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.138772 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.138730 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:33.180497 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.180431 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-federate-client-tls\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.180682 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.180508 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-metrics-client-ca\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.180682 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.180564 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-secret-telemeter-client\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.180682 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.180602 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.180682 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.180630 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-telemeter-client-tls\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.180682 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.180679 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-serving-certs-ca-bundle\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.181509 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.181093 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c42s\" (UniqueName: \"kubernetes.io/projected/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-kube-api-access-6c42s\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.181509 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.181183 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.181509 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.181367 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-serving-certs-ca-bundle\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.181509 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.181367 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-metrics-client-ca\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.182211 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.182152 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.183873 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.183847 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-federate-client-tls\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.183987 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.183875 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.183987 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.183968 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-telemeter-client-tls\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.184236 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.184212 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-secret-telemeter-client\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.192714 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.192656 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c42s\" (UniqueName: \"kubernetes.io/projected/8c47284b-8397-4f76-9cb9-b1ce93b25e8f-kube-api-access-6c42s\") pod \"telemeter-client-6455cb7df7-swbff\" (UID: \"8c47284b-8397-4f76-9cb9-b1ce93b25e8f\") " pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.252649 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.252614 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" Jul 06 19:00:33.280292 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.280258 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:33.525541 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.525496 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-749c6dcb9-sb5v4"] Jul 06 19:00:33.529345 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.529320 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.538341 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.538270 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 19:00:33.544206 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.544177 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-749c6dcb9-sb5v4"] Jul 06 19:00:33.585493 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.585458 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq25n\" (UniqueName: \"kubernetes.io/projected/ad18adb3-6395-4684-9017-f72ec4891cdc-kube-api-access-jq25n\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.585661 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.585579 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-trusted-ca-bundle\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.585742 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.585695 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad18adb3-6395-4684-9017-f72ec4891cdc-console-serving-cert\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.585742 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.585738 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad18adb3-6395-4684-9017-f72ec4891cdc-console-oauth-config\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.585855 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.585756 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-oauth-serving-cert\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.585855 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.585792 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-service-ca\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.585855 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.585836 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-console-config\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.686431 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.686390 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-service-ca\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.686625 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.686484 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-console-config\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.686625 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.686540 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq25n\" (UniqueName: \"kubernetes.io/projected/ad18adb3-6395-4684-9017-f72ec4891cdc-kube-api-access-jq25n\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.686625 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.686575 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-trusted-ca-bundle\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.686786 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.686651 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad18adb3-6395-4684-9017-f72ec4891cdc-console-serving-cert\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.686786 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.686692 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad18adb3-6395-4684-9017-f72ec4891cdc-console-oauth-config\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.686786 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.686717 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-oauth-serving-cert\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.687339 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.687285 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-service-ca\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.687537 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.687517 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-oauth-serving-cert\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.687730 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.687705 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-trusted-ca-bundle\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.688057 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.688039 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-console-config\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.689764 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.689743 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad18adb3-6395-4684-9017-f72ec4891cdc-console-oauth-config\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.689904 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.689883 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad18adb3-6395-4684-9017-f72ec4891cdc-console-serving-cert\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.707895 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.707867 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq25n\" (UniqueName: \"kubernetes.io/projected/ad18adb3-6395-4684-9017-f72ec4891cdc-kube-api-access-jq25n\") pod \"console-749c6dcb9-sb5v4\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.840881 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.840795 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:33.995054 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.995017 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:00:33.999227 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:33.999201 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.005123 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.005083 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 19:00:34.006928 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.006891 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3ek2knif8baik\"" Jul 06 19:00:34.006928 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.006912 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 19:00:34.007646 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.007626 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 19:00:34.007804 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.007782 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 19:00:34.007975 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.007738 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-45pqd\"" Jul 06 19:00:34.008201 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.007972 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 19:00:34.008632 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.008016 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 19:00:34.008895 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.008275 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 19:00:34.009102 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.009080 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 19:00:34.009940 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.009918 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 19:00:34.010041 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.009926 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 19:00:34.010041 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.009976 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 19:00:34.010555 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.010517 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 19:00:34.011278 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.011011 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 19:00:34.012321 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.012300 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 19:00:34.041986 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.040029 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:34.041986 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.040476 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:00:34.092966 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.092876 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.092966 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.092930 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-config-out\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.092966 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.092959 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093233 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.092993 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093233 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.093037 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093233 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.093061 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093233 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.093084 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093233 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.093119 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093233 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.093176 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093233 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.093223 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093489 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.093246 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093489 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.093295 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n79dg\" (UniqueName: \"kubernetes.io/projected/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-kube-api-access-n79dg\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093489 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.093336 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093489 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.093352 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093489 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.093368 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-config\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093489 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.093382 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093489 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.093397 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093489 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.093461 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093803 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.093494 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-web-config\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.093803 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.093521 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.194248 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194213 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.194248 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194244 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.194531 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194267 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.194531 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194285 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.194531 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194317 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n79dg\" (UniqueName: \"kubernetes.io/projected/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-kube-api-access-n79dg\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.194531 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194346 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.194531 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194363 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.194531 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194377 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-config\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.194531 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194393 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.194531 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194407 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.194531 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194426 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.194531 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194466 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-web-config\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.194531 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194492 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.194531 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194514 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.194531 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194533 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-config-out\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.195163 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194553 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.195163 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194574 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.195163 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194608 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.195163 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194622 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.195163 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.194640 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.195163 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.195149 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.195686 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.195654 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.196559 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.196533 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.196844 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.196822 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.197955 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.197621 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.201324 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.198676 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.201324 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.199341 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.201324 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.201195 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.201558 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.201382 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.201558 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.201494 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.201983 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.201895 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.202078 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.202047 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.203182 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.202738 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-config-out\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.203182 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.202833 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.203182 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.202994 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.203182 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.203115 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.203182 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.203144 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-config\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.203952 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.203918 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-web-config\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.204865 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.204819 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.206828 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.206806 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n79dg\" (UniqueName: \"kubernetes.io/projected/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-kube-api-access-n79dg\") pod \"prometheus-k8s-0\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.313887 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.313822 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:00:34.415480 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.415042 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:34.415480 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.415179 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:34.420406 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.420373 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:34.574890 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.574845 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:34.663592 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.663563 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:00:34.671822 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:34.671738 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:36.440167 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:36.440122 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:36.765167 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:36.765128 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:36.877136 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:36.877096 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:37.242191 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:37.242142 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:37.335084 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:37.335047 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:42.032879 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.032840 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:42.143020 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.142958 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:42.323184 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.323108 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-d84pd"] Jul 06 19:00:42.406834 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.406721 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:00:42.436267 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.436149 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:00:42.457230 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.457197 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-749c6dcb9-sb5v4"] Jul 06 19:00:42.525337 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.525234 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-57b7fccbbf-5c2nh"] Jul 06 19:00:42.526299 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.525842 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6455cb7df7-swbff"] Jul 06 19:00:42.635480 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.635360 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp"] Jul 06 19:00:42.639391 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:00:42.639274 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5abc161e_79ec_4ba0_b063_084f93dfe838.slice/crio-051d0f5378691d8d52ecad5183d4463313f06db64682e0e543055728151cc972 WatchSource:0}: Error finding container 051d0f5378691d8d52ecad5183d4463313f06db64682e0e543055728151cc972: Status 404 returned error can't find the container with id 051d0f5378691d8d52ecad5183d4463313f06db64682e0e543055728151cc972 Jul 06 19:00:42.691895 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.691787 2557 generic.go:356] "Generic (PLEG): container finished" podID="5e3dcc7d-c527-43c9-8d48-35fccf277853" containerID="1b9038bc532f1fbbacc08ad5ce2e241e53116022b36b88a982fc9c644f046124" exitCode=0 Jul 06 19:00:42.691895 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.691878 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vnsbs" event={"ID":"5e3dcc7d-c527-43c9-8d48-35fccf277853","Type":"ContainerDied","Data":"1b9038bc532f1fbbacc08ad5ce2e241e53116022b36b88a982fc9c644f046124"} Jul 06 19:00:42.693318 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.693276 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-d84pd" event={"ID":"5c8854d1-590f-4b85-8925-1cd1eb42dc13","Type":"ContainerStarted","Data":"18e52fd6fd92ee8c3d33fee344f22c19b4bd55ff1b69a37dc08b4780101e1a42"} Jul 06 19:00:42.694977 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.694948 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"36719a7e-f0d4-4c37-b625-d9e5bd3827d4","Type":"ContainerStarted","Data":"7fd9c423bbfa4e18b5ff00f56480deb93e6921380faa8a8a8ea2f928a3a5da04"} Jul 06 19:00:42.697666 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.697589 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-69gb8" event={"ID":"794f47cc-dce2-43e8-b571-d431f4b6cec0","Type":"ContainerStarted","Data":"893af24a05581cfd82da45768a2e0fb95ad7276b528defaf80d83b547dcd7f4f"} Jul 06 19:00:42.697764 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.697725 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 19:00:42.704430 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.704389 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749c6dcb9-sb5v4" event={"ID":"ad18adb3-6395-4684-9017-f72ec4891cdc","Type":"ContainerStarted","Data":"2ef90fda60b45e841b0683b8a9148ceec5116cbc3a439d6b26b11eb6b16f1ac4"} Jul 06 19:00:42.704430 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.704427 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749c6dcb9-sb5v4" event={"ID":"ad18adb3-6395-4684-9017-f72ec4891cdc","Type":"ContainerStarted","Data":"77ff7590e78bb9c5e8490bf20fddda4887c763be198cf974279255cbc06064f7"} Jul 06 19:00:42.706027 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.705984 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" event={"ID":"5abc161e-79ec-4ba0-b063-084f93dfe838","Type":"ContainerStarted","Data":"051d0f5378691d8d52ecad5183d4463313f06db64682e0e543055728151cc972"} Jul 06 19:00:42.707692 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.707625 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-xd58h" event={"ID":"bf18ee49-d386-429b-b848-ed10ff8f5280","Type":"ContainerStarted","Data":"d535b6f25205c13d9d3e4d5a7c4560824e4b34778b54d4fba1be7fa21dd27c20"} Jul 06 19:00:42.708323 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.708041 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-xd58h" Jul 06 19:00:42.710183 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.710162 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" event={"ID":"03b694d1-fffd-4360-867b-72db963a7a15","Type":"ContainerStarted","Data":"ff509d0f6f51bddd4b6735b51769f1fdd6db41855f980938199f0b32e879e524"} Jul 06 19:00:42.711577 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.711555 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" event={"ID":"8c47284b-8397-4f76-9cb9-b1ce93b25e8f","Type":"ContainerStarted","Data":"4a880d25be79bca0427795156b990c4dff913e4c24c7d040d002654b3efcab69"} Jul 06 19:00:42.713638 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.713601 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k9pnm" event={"ID":"04e067b8-9d48-42c6-8cac-0254ea3a4328","Type":"ContainerStarted","Data":"793e5df709038ba96cddabc4ecfa112608c7705f772c9c8df95bc5bff5a68fc1"} Jul 06 19:00:42.716633 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.716616 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-xd58h" Jul 06 19:00:42.719898 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.719256 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" event={"ID":"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153","Type":"ContainerStarted","Data":"a844525ab00ba29ad2d6c86152701daa324e31a23ab86b236f453d0ea345a18a"} Jul 06 19:00:42.719898 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.719305 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" event={"ID":"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153","Type":"ContainerStarted","Data":"62040f652d4628c7edb88fb4bd9b77eabc5814fec4ed008ba07988665f56d47b"} Jul 06 19:00:42.719898 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.719334 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" event={"ID":"48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153","Type":"ContainerStarted","Data":"2acbb0a4049e30c61e504e505b109d6380a55a42b88a15f641be4a8050cf1dce"} Jul 06 19:00:42.721909 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.721675 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e373d67-33bb-4271-8d89-0f4f45c612e7","Type":"ContainerStarted","Data":"f11217faa410e7956ea9850425109372c651429508776246128fbeca312f0050"} Jul 06 19:00:42.724474 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.724352 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" event={"ID":"77d08ee5-6984-47b3-8e6c-9645bc5afe52","Type":"ContainerStarted","Data":"ceb22a29e38baffda98e95fb926d584ae362f93a9231a6b47815e99480096265"} Jul 06 19:00:42.775557 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.775502 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-749c6dcb9-sb5v4" podStartSLOduration=9.775480254 podStartE2EDuration="9.775480254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:42.742953841 +0000 UTC m=+83.104117633" watchObservedRunningTime="2026-07-06 19:00:42.775480254 +0000 UTC m=+83.136644048" Jul 06 19:00:42.776528 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.776256 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-69gb8" podStartSLOduration=67.129713766 podStartE2EDuration="1m22.776238926s" totalImagesPullingTime="15.64652516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:42.776230169 +0000 UTC m=+83.137393952" watchObservedRunningTime="2026-07-06 19:00:42.776238926 +0000 UTC m=+83.137402720" Jul 06 19:00:42.810607 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.810552 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k9pnm" podStartSLOduration=67.129452329 podStartE2EDuration="1m22.810532567s" totalImagesPullingTime="15.681080238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:42.809520004 +0000 UTC m=+83.170683799" watchObservedRunningTime="2026-07-06 19:00:42.810532567 +0000 UTC m=+83.171696363" Jul 06 19:00:42.835218 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.835151 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-xd58h" podStartSLOduration=1.457852705 podStartE2EDuration="28.835126784s" totalImagesPullingTime="27.377274079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:42.833589159 +0000 UTC m=+83.194752951" watchObservedRunningTime="2026-07-06 19:00:42.835126784 +0000 UTC m=+83.196290577" Jul 06 19:00:42.866661 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.866596 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gsfgd" podStartSLOduration=3.442673652 podStartE2EDuration="15.866572793s" totalImagesPullingTime="12.423899141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:42.862835848 +0000 UTC m=+83.223999644" watchObservedRunningTime="2026-07-06 19:00:42.866572793 +0000 UTC m=+83.227736586" Jul 06 19:00:42.910300 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:42.910237 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-9j6j7" podStartSLOduration=3.643221906 podStartE2EDuration="15.910216555s" totalImagesPullingTime="12.266994649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:42.909246473 +0000 UTC m=+83.270410305" watchObservedRunningTime="2026-07-06 19:00:42.910216555 +0000 UTC m=+83.271380349" Jul 06 19:00:43.746109 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:43.746067 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vnsbs" event={"ID":"5e3dcc7d-c527-43c9-8d48-35fccf277853","Type":"ContainerStarted","Data":"5739d58f94c6ebc5aea070c47d71565b33f40c603ffde951ec2016390d468fdc"} Jul 06 19:00:43.747571 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:43.746836 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vnsbs" event={"ID":"5e3dcc7d-c527-43c9-8d48-35fccf277853","Type":"ContainerStarted","Data":"0fa0b8b8c65f01c409180f5ae3d8f709c9a674e3ea04b6e679265de9cb625c47"} Jul 06 19:00:43.842731 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:43.842219 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:43.842731 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:43.842417 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:43.850022 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:43.849787 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:43.878932 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:43.876151 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vnsbs" podStartSLOduration=4.082274385 podStartE2EDuration="16.876115224s" totalImagesPullingTime="12.618313733s" totalInitContainerRuntime="175.527106ms" isStatefulPod=true podCreationTimestamp="2026-07-06 19:00:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:43.779514227 +0000 UTC m=+84.140678019" watchObservedRunningTime="2026-07-06 19:00:43.876115224 +0000 UTC m=+84.237279016" Jul 06 19:00:44.758214 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:44.758065 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:00:44.851783 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:44.851749 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-696dc88d99-mqwl4"] Jul 06 19:00:45.556774 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:45.556727 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:45.915011 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:45.914699 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:46.799339 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:46.799299 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:46.927673 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:46.927631 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:47.109192 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:47.109098 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:47.960413 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:47.960374 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:48.214015 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:48.213934 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:48.795030 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:48.794980 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" event={"ID":"5abc161e-79ec-4ba0-b063-084f93dfe838","Type":"ContainerStarted","Data":"4edf7b44811bf5d3c472260df1e877bff1a6bb6841f0f8b4661e7d0361b3c225"} Jul 06 19:00:48.795203 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:48.795099 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" event={"ID":"5abc161e-79ec-4ba0-b063-084f93dfe838","Type":"ContainerStarted","Data":"e4e52c6aec55f28bad67b476e4101054678ffe01ba42af9b4406403e2fb263b7"} Jul 06 19:00:48.795203 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:48.795146 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" event={"ID":"5abc161e-79ec-4ba0-b063-084f93dfe838","Type":"ContainerStarted","Data":"eae3576e5502c7353959a1635b6065bd31647ad57ca1e144809567db88cde77c"} Jul 06 19:00:48.798685 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:48.798652 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" event={"ID":"03b694d1-fffd-4360-867b-72db963a7a15","Type":"ContainerStarted","Data":"72527a84605f92e74401261c22ebfdae53fa429348fe6ff7cfbad2c8470ba504"} Jul 06 19:00:48.801712 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:48.801603 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" event={"ID":"8c47284b-8397-4f76-9cb9-b1ce93b25e8f","Type":"ContainerStarted","Data":"d84de9c335d21e7833b02a07304689fca494259c8c51cdb1002e49bb6a07dcc7"} Jul 06 19:00:48.801712 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:48.801667 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" event={"ID":"8c47284b-8397-4f76-9cb9-b1ce93b25e8f","Type":"ContainerStarted","Data":"a8be56141fd2038d96cc970ceaa59284be3e7bfbfeb407680b13247e7839773a"} Jul 06 19:00:48.803552 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:48.803529 2557 generic.go:356] "Generic (PLEG): container finished" podID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerID="5fd3f8e759dd991da4f49636a958d12d689b2cc46e181f75f465af7ad9b56eac" exitCode=0 Jul 06 19:00:48.803761 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:48.803599 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e373d67-33bb-4271-8d89-0f4f45c612e7","Type":"ContainerDied","Data":"5fd3f8e759dd991da4f49636a958d12d689b2cc46e181f75f465af7ad9b56eac"} Jul 06 19:00:48.807756 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:48.806129 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-d84pd" event={"ID":"5c8854d1-590f-4b85-8925-1cd1eb42dc13","Type":"ContainerStarted","Data":"08f0dfc75c92811717ec32c482d27fc1b60461c2ff5409937ddb408deca141a1"} Jul 06 19:00:48.809953 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:48.808251 2557 generic.go:356] "Generic (PLEG): container finished" podID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerID="f4b543433a6c9b0e42c702595e979b49efbb7e1a2490249ac9b363736d00c8b8" exitCode=0 Jul 06 19:00:48.809953 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:48.809022 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"36719a7e-f0d4-4c37-b625-d9e5bd3827d4","Type":"ContainerDied","Data":"f4b543433a6c9b0e42c702595e979b49efbb7e1a2490249ac9b363736d00c8b8"} Jul 06 19:00:48.809953 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:48.809087 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-d84pd" Jul 06 19:00:48.816695 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:48.816632 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-d84pd" Jul 06 19:00:48.827182 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:48.826802 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" podStartSLOduration=11.058892815 podStartE2EDuration="16.826782092s" totalImagesPullingTime="5.767889277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:48.825327917 +0000 UTC m=+89.186491710" watchObservedRunningTime="2026-07-06 19:00:48.826782092 +0000 UTC m=+89.187945891" Jul 06 19:00:48.850058 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:48.849999 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-d84pd" podStartSLOduration=10.857434854 podStartE2EDuration="16.849978451s" totalImagesPullingTime="5.992543597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:48.849412573 +0000 UTC m=+89.210576370" watchObservedRunningTime="2026-07-06 19:00:48.849978451 +0000 UTC m=+89.211142221" Jul 06 19:00:49.821276 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:49.821181 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" event={"ID":"8c47284b-8397-4f76-9cb9-b1ce93b25e8f","Type":"ContainerStarted","Data":"7cc9d4f683de3260584e3119c20a8b8a1b30977419122041098ca67583755ad0"} Jul 06 19:00:49.847665 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:49.847573 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6455cb7df7-swbff" podStartSLOduration=12.071896231 podStartE2EDuration="17.847547298s" totalImagesPullingTime="5.775651067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:49.846845782 +0000 UTC m=+90.208009576" watchObservedRunningTime="2026-07-06 19:00:49.847547298 +0000 UTC m=+90.208711093" Jul 06 19:00:50.832961 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:50.832906 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" event={"ID":"5abc161e-79ec-4ba0-b063-084f93dfe838","Type":"ContainerStarted","Data":"1c6bf76cf11059a44347d7669ab2186de8480ba0abdcf449117735bf9f8f1e12"} Jul 06 19:00:51.093740 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:51.093575 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:51.198292 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:51.198262 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:51.840974 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:51.840939 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e373d67-33bb-4271-8d89-0f4f45c612e7","Type":"ContainerStarted","Data":"9493ee6e84a3b6ba955245b86ae4a09dfa373008738176f29f0007aa559b352a"} Jul 06 19:00:51.841391 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:51.841021 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e373d67-33bb-4271-8d89-0f4f45c612e7","Type":"ContainerStarted","Data":"df880413895d798542d49994722d567bc4740784029e22c2f07971941e870a0d"} Jul 06 19:00:51.844426 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:51.844402 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" event={"ID":"5abc161e-79ec-4ba0-b063-084f93dfe838","Type":"ContainerStarted","Data":"d80c44b00d635c749cbbb345b8d910a3076748cd922aef2eb9bdb7e2217c3a25"} Jul 06 19:00:51.844580 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:51.844493 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" event={"ID":"5abc161e-79ec-4ba0-b063-084f93dfe838","Type":"ContainerStarted","Data":"73b32c2336dcf17e8dabf7cda439e46ee65066842c34dbee38fb8869cd48dfa8"} Jul 06 19:00:51.856070 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:51.856042 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:51.883793 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:51.883733 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" podStartSLOduration=14.553318624 podStartE2EDuration="21.883711575s" totalImagesPullingTime="7.330392951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:51.879996111 +0000 UTC m=+92.241159905" watchObservedRunningTime="2026-07-06 19:00:51.883711575 +0000 UTC m=+92.244875368" Jul 06 19:00:52.401702 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:52.401664 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:52.401895 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:52.401739 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:00:52.852487 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:52.852427 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e373d67-33bb-4271-8d89-0f4f45c612e7","Type":"ContainerStarted","Data":"b182e3f7f38f26ac9bfab64c49c99272bc3aaf897d819a3b714900eff7611f9b"} Jul 06 19:00:53.860802 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:53.860768 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e373d67-33bb-4271-8d89-0f4f45c612e7","Type":"ContainerStarted","Data":"fb7e991159986380b7b5d9f0a536eb216f60fe47f50627b6c53a588a65fa6072"} Jul 06 19:00:53.861174 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:53.860893 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e373d67-33bb-4271-8d89-0f4f45c612e7","Type":"ContainerStarted","Data":"31c676b9bd07363f676e22a42fc54b774f939f4a398d489ebd44ffe315196334"} Jul 06 19:00:53.861174 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:53.860923 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e373d67-33bb-4271-8d89-0f4f45c612e7","Type":"ContainerStarted","Data":"96cf2e67ee051bd7a24fbce22b142877aa2de6346b28de9d6a66f8dd20ee9f9a"} Jul 06 19:00:53.863132 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:53.863110 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"36719a7e-f0d4-4c37-b625-d9e5bd3827d4","Type":"ContainerStarted","Data":"8a73ea8ad43a4ccd68d525bb1f44087219a04505e017a4d02f1bf05bf6cdd887"} Jul 06 19:00:53.863221 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:53.863163 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"36719a7e-f0d4-4c37-b625-d9e5bd3827d4","Type":"ContainerStarted","Data":"31c8e5d8b9305f32bb0170c1be1467bc4fbd1690c38065ae9f95ecd8fe2f5ac5"} Jul 06 19:00:53.892966 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:53.892888 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=17.267783733 podStartE2EDuration="25.892863741s" totalImagesPullingTime="8.621525604s" totalInitContainerRuntime="3.554404ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:53.889519534 +0000 UTC m=+94.250683326" watchObservedRunningTime="2026-07-06 19:00:53.892863741 +0000 UTC m=+94.254027534" Jul 06 19:00:54.870667 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:54.870626 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"36719a7e-f0d4-4c37-b625-d9e5bd3827d4","Type":"ContainerStarted","Data":"475a207ac7217978cb49c29aae51a0d409d3bff74ba4907c88c0c40b083632ce"} Jul 06 19:00:54.871157 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:54.870715 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"36719a7e-f0d4-4c37-b625-d9e5bd3827d4","Type":"ContainerStarted","Data":"87c3175e1027d269c648925eeed63f214d70cce9936916da3e35035cb63937cd"} Jul 06 19:00:54.871157 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:54.870748 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"36719a7e-f0d4-4c37-b625-d9e5bd3827d4","Type":"ContainerStarted","Data":"0c75f3d09ff3a894bc63d573e3959bc2763099a84935f233fcf2dd0f55036567"} Jul 06 19:00:54.871157 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:54.870781 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"36719a7e-f0d4-4c37-b625-d9e5bd3827d4","Type":"ContainerStarted","Data":"f1189e098046c182b337e3ef5cf5b92623e18c3cbdd896ff6e56618d3871df80"} Jul 06 19:00:54.906579 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:54.906515 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=11.298017171 podStartE2EDuration="21.906491488s" totalImagesPullingTime="10.578627607s" totalInitContainerRuntime="29.84671ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:54.904297975 +0000 UTC m=+95.265461768" watchObservedRunningTime="2026-07-06 19:00:54.906491488 +0000 UTC m=+95.267655281" Jul 06 19:00:57.858060 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:57.858028 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6b85fc5cc7-wpfrp" Jul 06 19:00:59.314239 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:00:59.314201 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:01.595288 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:01.595252 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-749c6dcb9-sb5v4"] Jul 06 19:01:01.747279 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:01.747245 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:01.865917 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:01.865834 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:01.929778 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:01.929741 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:02.001339 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:02.001296 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:03.193692 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:03.193654 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:03.317177 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:03.317141 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:09.887720 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:09.887674 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-696dc88d99-mqwl4" podUID="5c125efd-4ffd-4d90-ac8a-ea2133b24143" containerName="console" containerID="cri-o://65411cbf14e99e9f36e37dc88750eb0693455161fc1b2b2fca08f98a2fe89408" gracePeriod=15 Jul 06 19:01:10.178836 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.178805 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:01:10.286248 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.286213 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8bkl\" (UniqueName: \"kubernetes.io/projected/5c125efd-4ffd-4d90-ac8a-ea2133b24143-kube-api-access-f8bkl\") pod \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " Jul 06 19:01:10.286248 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.286254 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-config\") pod \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " Jul 06 19:01:10.286540 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.286279 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-oauth-serving-cert\") pod \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " Jul 06 19:01:10.286540 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.286305 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-serving-cert\") pod \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " Jul 06 19:01:10.286540 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.286476 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-service-ca\") pod \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " Jul 06 19:01:10.286704 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.286559 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-oauth-config\") pod \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\" (UID: \"5c125efd-4ffd-4d90-ac8a-ea2133b24143\") " Jul 06 19:01:10.286759 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.286736 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-config" (OuterVolumeSpecName: "console-config") pod "5c125efd-4ffd-4d90-ac8a-ea2133b24143" (UID: "5c125efd-4ffd-4d90-ac8a-ea2133b24143"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:01:10.286813 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.286777 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5c125efd-4ffd-4d90-ac8a-ea2133b24143" (UID: "5c125efd-4ffd-4d90-ac8a-ea2133b24143"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:01:10.286897 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.286849 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-service-ca" (OuterVolumeSpecName: "service-ca") pod "5c125efd-4ffd-4d90-ac8a-ea2133b24143" (UID: "5c125efd-4ffd-4d90-ac8a-ea2133b24143"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:01:10.287050 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.287029 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-config\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:10.287118 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.287054 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-oauth-serving-cert\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:10.287118 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.287070 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c125efd-4ffd-4d90-ac8a-ea2133b24143-service-ca\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:10.288830 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.288802 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c125efd-4ffd-4d90-ac8a-ea2133b24143-kube-api-access-f8bkl" (OuterVolumeSpecName: "kube-api-access-f8bkl") pod "5c125efd-4ffd-4d90-ac8a-ea2133b24143" (UID: "5c125efd-4ffd-4d90-ac8a-ea2133b24143"). InnerVolumeSpecName "kube-api-access-f8bkl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:01:10.288917 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.288800 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5c125efd-4ffd-4d90-ac8a-ea2133b24143" (UID: "5c125efd-4ffd-4d90-ac8a-ea2133b24143"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:10.288917 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.288850 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5c125efd-4ffd-4d90-ac8a-ea2133b24143" (UID: "5c125efd-4ffd-4d90-ac8a-ea2133b24143"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:10.388170 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.388119 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8bkl\" (UniqueName: \"kubernetes.io/projected/5c125efd-4ffd-4d90-ac8a-ea2133b24143-kube-api-access-f8bkl\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:10.388170 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.388164 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-serving-cert\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:10.388170 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.388177 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c125efd-4ffd-4d90-ac8a-ea2133b24143-console-oauth-config\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:10.942658 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.942620 2557 generic.go:356] "Generic (PLEG): container finished" podID="5c125efd-4ffd-4d90-ac8a-ea2133b24143" containerID="65411cbf14e99e9f36e37dc88750eb0693455161fc1b2b2fca08f98a2fe89408" exitCode=0 Jul 06 19:01:10.943123 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.942682 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696dc88d99-mqwl4" Jul 06 19:01:10.943123 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.942685 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696dc88d99-mqwl4" event={"ID":"5c125efd-4ffd-4d90-ac8a-ea2133b24143","Type":"ContainerDied","Data":"65411cbf14e99e9f36e37dc88750eb0693455161fc1b2b2fca08f98a2fe89408"} Jul 06 19:01:10.943123 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.942835 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696dc88d99-mqwl4" event={"ID":"5c125efd-4ffd-4d90-ac8a-ea2133b24143","Type":"ContainerDied","Data":"0b529cf7879dc66582ef611e888e6ed61a65bbeb82c216beb9eeaab927238358"} Jul 06 19:01:10.943123 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.942866 2557 scope.go:117] "RemoveContainer" containerID="65411cbf14e99e9f36e37dc88750eb0693455161fc1b2b2fca08f98a2fe89408" Jul 06 19:01:10.952141 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.952114 2557 scope.go:117] "RemoveContainer" containerID="65411cbf14e99e9f36e37dc88750eb0693455161fc1b2b2fca08f98a2fe89408" Jul 06 19:01:10.952476 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:01:10.952450 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65411cbf14e99e9f36e37dc88750eb0693455161fc1b2b2fca08f98a2fe89408\": container with ID starting with 65411cbf14e99e9f36e37dc88750eb0693455161fc1b2b2fca08f98a2fe89408 not found: ID does not exist" containerID="65411cbf14e99e9f36e37dc88750eb0693455161fc1b2b2fca08f98a2fe89408" Jul 06 19:01:10.952534 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.952487 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65411cbf14e99e9f36e37dc88750eb0693455161fc1b2b2fca08f98a2fe89408"} err="failed to get container status \"65411cbf14e99e9f36e37dc88750eb0693455161fc1b2b2fca08f98a2fe89408\": rpc error: code = NotFound desc = could not find container \"65411cbf14e99e9f36e37dc88750eb0693455161fc1b2b2fca08f98a2fe89408\": container with ID starting with 65411cbf14e99e9f36e37dc88750eb0693455161fc1b2b2fca08f98a2fe89408 not found: ID does not exist" Jul 06 19:01:10.971179 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.971148 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-696dc88d99-mqwl4"] Jul 06 19:01:10.976649 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:10.976625 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-696dc88d99-mqwl4"] Jul 06 19:01:12.196259 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:12.196227 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c125efd-4ffd-4d90-ac8a-ea2133b24143" path="/var/lib/kubelet/pods/5c125efd-4ffd-4d90-ac8a-ea2133b24143/volumes" Jul 06 19:01:12.406599 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:12.406570 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:01:12.410332 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:12.410309 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-57b7fccbbf-5c2nh" Jul 06 19:01:13.747956 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:13.747922 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-69gb8" Jul 06 19:01:26.616361 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:26.616286 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-749c6dcb9-sb5v4" podUID="ad18adb3-6395-4684-9017-f72ec4891cdc" containerName="console" containerID="cri-o://2ef90fda60b45e841b0683b8a9148ceec5116cbc3a439d6b26b11eb6b16f1ac4" gracePeriod=15 Jul 06 19:01:26.904291 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:26.904264 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:01:27.019776 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.019738 2557 generic.go:356] "Generic (PLEG): container finished" podID="ad18adb3-6395-4684-9017-f72ec4891cdc" containerID="2ef90fda60b45e841b0683b8a9148ceec5116cbc3a439d6b26b11eb6b16f1ac4" exitCode=0 Jul 06 19:01:27.019974 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.019786 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749c6dcb9-sb5v4" event={"ID":"ad18adb3-6395-4684-9017-f72ec4891cdc","Type":"ContainerDied","Data":"2ef90fda60b45e841b0683b8a9148ceec5116cbc3a439d6b26b11eb6b16f1ac4"} Jul 06 19:01:27.019974 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.019795 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749c6dcb9-sb5v4" Jul 06 19:01:27.019974 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.019829 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749c6dcb9-sb5v4" event={"ID":"ad18adb3-6395-4684-9017-f72ec4891cdc","Type":"ContainerDied","Data":"77ff7590e78bb9c5e8490bf20fddda4887c763be198cf974279255cbc06064f7"} Jul 06 19:01:27.019974 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.019851 2557 scope.go:117] "RemoveContainer" containerID="2ef90fda60b45e841b0683b8a9148ceec5116cbc3a439d6b26b11eb6b16f1ac4" Jul 06 19:01:27.027909 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.027892 2557 scope.go:117] "RemoveContainer" containerID="2ef90fda60b45e841b0683b8a9148ceec5116cbc3a439d6b26b11eb6b16f1ac4" Jul 06 19:01:27.028188 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:01:27.028168 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ef90fda60b45e841b0683b8a9148ceec5116cbc3a439d6b26b11eb6b16f1ac4\": container with ID starting with 2ef90fda60b45e841b0683b8a9148ceec5116cbc3a439d6b26b11eb6b16f1ac4 not found: ID does not exist" containerID="2ef90fda60b45e841b0683b8a9148ceec5116cbc3a439d6b26b11eb6b16f1ac4" Jul 06 19:01:27.028245 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.028197 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ef90fda60b45e841b0683b8a9148ceec5116cbc3a439d6b26b11eb6b16f1ac4"} err="failed to get container status \"2ef90fda60b45e841b0683b8a9148ceec5116cbc3a439d6b26b11eb6b16f1ac4\": rpc error: code = NotFound desc = could not find container \"2ef90fda60b45e841b0683b8a9148ceec5116cbc3a439d6b26b11eb6b16f1ac4\": container with ID starting with 2ef90fda60b45e841b0683b8a9148ceec5116cbc3a439d6b26b11eb6b16f1ac4 not found: ID does not exist" Jul 06 19:01:27.044555 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.044530 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-oauth-serving-cert\") pod \"ad18adb3-6395-4684-9017-f72ec4891cdc\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " Jul 06 19:01:27.044653 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.044583 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-service-ca\") pod \"ad18adb3-6395-4684-9017-f72ec4891cdc\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " Jul 06 19:01:27.044653 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.044625 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-trusted-ca-bundle\") pod \"ad18adb3-6395-4684-9017-f72ec4891cdc\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " Jul 06 19:01:27.044653 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.044651 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad18adb3-6395-4684-9017-f72ec4891cdc-console-oauth-config\") pod \"ad18adb3-6395-4684-9017-f72ec4891cdc\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " Jul 06 19:01:27.044798 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.044671 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jq25n\" (UniqueName: \"kubernetes.io/projected/ad18adb3-6395-4684-9017-f72ec4891cdc-kube-api-access-jq25n\") pod \"ad18adb3-6395-4684-9017-f72ec4891cdc\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " Jul 06 19:01:27.044798 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.044698 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-console-config\") pod \"ad18adb3-6395-4684-9017-f72ec4891cdc\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " Jul 06 19:01:27.044798 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.044739 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad18adb3-6395-4684-9017-f72ec4891cdc-console-serving-cert\") pod \"ad18adb3-6395-4684-9017-f72ec4891cdc\" (UID: \"ad18adb3-6395-4684-9017-f72ec4891cdc\") " Jul 06 19:01:27.045025 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.044996 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ad18adb3-6395-4684-9017-f72ec4891cdc" (UID: "ad18adb3-6395-4684-9017-f72ec4891cdc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:01:27.045144 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.045065 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ad18adb3-6395-4684-9017-f72ec4891cdc" (UID: "ad18adb3-6395-4684-9017-f72ec4891cdc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:01:27.045205 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.045144 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-console-config" (OuterVolumeSpecName: "console-config") pod "ad18adb3-6395-4684-9017-f72ec4891cdc" (UID: "ad18adb3-6395-4684-9017-f72ec4891cdc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:01:27.045244 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.045223 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-service-ca" (OuterVolumeSpecName: "service-ca") pod "ad18adb3-6395-4684-9017-f72ec4891cdc" (UID: "ad18adb3-6395-4684-9017-f72ec4891cdc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:01:27.046963 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.046936 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad18adb3-6395-4684-9017-f72ec4891cdc-kube-api-access-jq25n" (OuterVolumeSpecName: "kube-api-access-jq25n") pod "ad18adb3-6395-4684-9017-f72ec4891cdc" (UID: "ad18adb3-6395-4684-9017-f72ec4891cdc"). InnerVolumeSpecName "kube-api-access-jq25n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:01:27.047052 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.046943 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad18adb3-6395-4684-9017-f72ec4891cdc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ad18adb3-6395-4684-9017-f72ec4891cdc" (UID: "ad18adb3-6395-4684-9017-f72ec4891cdc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:27.047052 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.047017 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad18adb3-6395-4684-9017-f72ec4891cdc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ad18adb3-6395-4684-9017-f72ec4891cdc" (UID: "ad18adb3-6395-4684-9017-f72ec4891cdc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:27.146187 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.146094 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-oauth-serving-cert\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:27.146187 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.146128 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-service-ca\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:27.146187 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.146137 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-trusted-ca-bundle\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:27.146187 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.146146 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad18adb3-6395-4684-9017-f72ec4891cdc-console-oauth-config\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:27.146187 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.146154 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jq25n\" (UniqueName: \"kubernetes.io/projected/ad18adb3-6395-4684-9017-f72ec4891cdc-kube-api-access-jq25n\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:27.146187 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.146163 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad18adb3-6395-4684-9017-f72ec4891cdc-console-config\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:27.146187 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.146171 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad18adb3-6395-4684-9017-f72ec4891cdc-console-serving-cert\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:27.346397 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.346363 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-749c6dcb9-sb5v4"] Jul 06 19:01:27.349254 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:27.349224 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-749c6dcb9-sb5v4"] Jul 06 19:01:28.195734 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:28.195701 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad18adb3-6395-4684-9017-f72ec4891cdc" path="/var/lib/kubelet/pods/ad18adb3-6395-4684-9017-f72ec4891cdc/volumes" Jul 06 19:01:34.314922 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:34.314884 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:34.380782 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:34.380752 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:35.069600 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:35.069572 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:43.944281 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:43.944242 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:44.044095 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:44.044058 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:48.602684 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:48.602654 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:01:48.603112 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:48.602935 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="alertmanager" containerID="cri-o://df880413895d798542d49994722d567bc4740784029e22c2f07971941e870a0d" gracePeriod=120 Jul 06 19:01:48.603112 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:48.603084 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="config-reloader" containerID="cri-o://9493ee6e84a3b6ba955245b86ae4a09dfa373008738176f29f0007aa559b352a" gracePeriod=120 Jul 06 19:01:48.603112 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:48.603091 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="prom-label-proxy" containerID="cri-o://fb7e991159986380b7b5d9f0a536eb216f60fe47f50627b6c53a588a65fa6072" gracePeriod=120 Jul 06 19:01:48.603291 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:48.603084 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="kube-rbac-proxy" containerID="cri-o://96cf2e67ee051bd7a24fbce22b142877aa2de6346b28de9d6a66f8dd20ee9f9a" gracePeriod=120 Jul 06 19:01:48.603291 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:48.603023 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="kube-rbac-proxy-metric" containerID="cri-o://31c676b9bd07363f676e22a42fc54b774f939f4a398d489ebd44ffe315196334" gracePeriod=120 Jul 06 19:01:48.603291 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:48.603084 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="kube-rbac-proxy-web" containerID="cri-o://b182e3f7f38f26ac9bfab64c49c99272bc3aaf897d819a3b714900eff7611f9b" gracePeriod=120 Jul 06 19:01:49.110754 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.110721 2557 generic.go:356] "Generic (PLEG): container finished" podID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerID="fb7e991159986380b7b5d9f0a536eb216f60fe47f50627b6c53a588a65fa6072" exitCode=0 Jul 06 19:01:49.110754 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.110747 2557 generic.go:356] "Generic (PLEG): container finished" podID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerID="31c676b9bd07363f676e22a42fc54b774f939f4a398d489ebd44ffe315196334" exitCode=0 Jul 06 19:01:49.110754 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.110754 2557 generic.go:356] "Generic (PLEG): container finished" podID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerID="96cf2e67ee051bd7a24fbce22b142877aa2de6346b28de9d6a66f8dd20ee9f9a" exitCode=0 Jul 06 19:01:49.110754 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.110761 2557 generic.go:356] "Generic (PLEG): container finished" podID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerID="9493ee6e84a3b6ba955245b86ae4a09dfa373008738176f29f0007aa559b352a" exitCode=0 Jul 06 19:01:49.111019 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.110767 2557 generic.go:356] "Generic (PLEG): container finished" podID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerID="df880413895d798542d49994722d567bc4740784029e22c2f07971941e870a0d" exitCode=0 Jul 06 19:01:49.111019 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.110790 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e373d67-33bb-4271-8d89-0f4f45c612e7","Type":"ContainerDied","Data":"fb7e991159986380b7b5d9f0a536eb216f60fe47f50627b6c53a588a65fa6072"} Jul 06 19:01:49.111019 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.110854 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e373d67-33bb-4271-8d89-0f4f45c612e7","Type":"ContainerDied","Data":"31c676b9bd07363f676e22a42fc54b774f939f4a398d489ebd44ffe315196334"} Jul 06 19:01:49.111019 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.110880 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e373d67-33bb-4271-8d89-0f4f45c612e7","Type":"ContainerDied","Data":"96cf2e67ee051bd7a24fbce22b142877aa2de6346b28de9d6a66f8dd20ee9f9a"} Jul 06 19:01:49.111019 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.110904 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e373d67-33bb-4271-8d89-0f4f45c612e7","Type":"ContainerDied","Data":"9493ee6e84a3b6ba955245b86ae4a09dfa373008738176f29f0007aa559b352a"} Jul 06 19:01:49.111019 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.110928 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e373d67-33bb-4271-8d89-0f4f45c612e7","Type":"ContainerDied","Data":"df880413895d798542d49994722d567bc4740784029e22c2f07971941e870a0d"} Jul 06 19:01:49.848073 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.848047 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:49.930002 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.929929 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-web-config\") pod \"1e373d67-33bb-4271-8d89-0f4f45c612e7\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " Jul 06 19:01:49.930970 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.930017 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1e373d67-33bb-4271-8d89-0f4f45c612e7-config-out\") pod \"1e373d67-33bb-4271-8d89-0f4f45c612e7\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " Jul 06 19:01:49.930970 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.930048 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e373d67-33bb-4271-8d89-0f4f45c612e7-metrics-client-ca\") pod \"1e373d67-33bb-4271-8d89-0f4f45c612e7\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " Jul 06 19:01:49.930970 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.930088 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e373d67-33bb-4271-8d89-0f4f45c612e7-alertmanager-trusted-ca-bundle\") pod \"1e373d67-33bb-4271-8d89-0f4f45c612e7\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " Jul 06 19:01:49.930970 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.930140 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy\") pod \"1e373d67-33bb-4271-8d89-0f4f45c612e7\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " Jul 06 19:01:49.930970 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.930172 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlpx2\" (UniqueName: \"kubernetes.io/projected/1e373d67-33bb-4271-8d89-0f4f45c612e7-kube-api-access-xlpx2\") pod \"1e373d67-33bb-4271-8d89-0f4f45c612e7\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " Jul 06 19:01:49.930970 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.930194 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-config-volume\") pod \"1e373d67-33bb-4271-8d89-0f4f45c612e7\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " Jul 06 19:01:49.930970 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.930234 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-cluster-tls-config\") pod \"1e373d67-33bb-4271-8d89-0f4f45c612e7\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " Jul 06 19:01:49.930970 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.930268 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-main-tls\") pod \"1e373d67-33bb-4271-8d89-0f4f45c612e7\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " Jul 06 19:01:49.930970 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.930297 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1e373d67-33bb-4271-8d89-0f4f45c612e7-alertmanager-main-db\") pod \"1e373d67-33bb-4271-8d89-0f4f45c612e7\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " Jul 06 19:01:49.930970 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.930345 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy-web\") pod \"1e373d67-33bb-4271-8d89-0f4f45c612e7\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " Jul 06 19:01:49.930970 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.930372 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1e373d67-33bb-4271-8d89-0f4f45c612e7-tls-assets\") pod \"1e373d67-33bb-4271-8d89-0f4f45c612e7\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " Jul 06 19:01:49.930970 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.930400 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1e373d67-33bb-4271-8d89-0f4f45c612e7\" (UID: \"1e373d67-33bb-4271-8d89-0f4f45c612e7\") " Jul 06 19:01:49.930970 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.930804 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e373d67-33bb-4271-8d89-0f4f45c612e7-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1e373d67-33bb-4271-8d89-0f4f45c612e7" (UID: "1e373d67-33bb-4271-8d89-0f4f45c612e7"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:01:49.930970 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.930887 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e373d67-33bb-4271-8d89-0f4f45c612e7-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1e373d67-33bb-4271-8d89-0f4f45c612e7" (UID: "1e373d67-33bb-4271-8d89-0f4f45c612e7"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:01:49.932002 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.931780 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e373d67-33bb-4271-8d89-0f4f45c612e7-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1e373d67-33bb-4271-8d89-0f4f45c612e7" (UID: "1e373d67-33bb-4271-8d89-0f4f45c612e7"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:01:49.935559 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.935483 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e373d67-33bb-4271-8d89-0f4f45c612e7-config-out" (OuterVolumeSpecName: "config-out") pod "1e373d67-33bb-4271-8d89-0f4f45c612e7" (UID: "1e373d67-33bb-4271-8d89-0f4f45c612e7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:01:49.935559 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.935499 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e373d67-33bb-4271-8d89-0f4f45c612e7-kube-api-access-xlpx2" (OuterVolumeSpecName: "kube-api-access-xlpx2") pod "1e373d67-33bb-4271-8d89-0f4f45c612e7" (UID: "1e373d67-33bb-4271-8d89-0f4f45c612e7"). InnerVolumeSpecName "kube-api-access-xlpx2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:01:49.935559 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.935532 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1e373d67-33bb-4271-8d89-0f4f45c612e7" (UID: "1e373d67-33bb-4271-8d89-0f4f45c612e7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:49.935798 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.935639 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "1e373d67-33bb-4271-8d89-0f4f45c612e7" (UID: "1e373d67-33bb-4271-8d89-0f4f45c612e7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:49.936091 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.936053 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1e373d67-33bb-4271-8d89-0f4f45c612e7" (UID: "1e373d67-33bb-4271-8d89-0f4f45c612e7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:49.936293 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.936248 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-config-volume" (OuterVolumeSpecName: "config-volume") pod "1e373d67-33bb-4271-8d89-0f4f45c612e7" (UID: "1e373d67-33bb-4271-8d89-0f4f45c612e7"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:49.936453 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.936405 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1e373d67-33bb-4271-8d89-0f4f45c612e7" (UID: "1e373d67-33bb-4271-8d89-0f4f45c612e7"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:49.936696 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.936678 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e373d67-33bb-4271-8d89-0f4f45c612e7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1e373d67-33bb-4271-8d89-0f4f45c612e7" (UID: "1e373d67-33bb-4271-8d89-0f4f45c612e7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:01:49.941146 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.941118 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "1e373d67-33bb-4271-8d89-0f4f45c612e7" (UID: "1e373d67-33bb-4271-8d89-0f4f45c612e7"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:49.948278 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:49.948248 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-web-config" (OuterVolumeSpecName: "web-config") pod "1e373d67-33bb-4271-8d89-0f4f45c612e7" (UID: "1e373d67-33bb-4271-8d89-0f4f45c612e7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:50.031111 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.031070 2557 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1e373d67-33bb-4271-8d89-0f4f45c612e7-config-out\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:50.031111 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.031105 2557 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e373d67-33bb-4271-8d89-0f4f45c612e7-metrics-client-ca\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:50.031111 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.031116 2557 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e373d67-33bb-4271-8d89-0f4f45c612e7-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:50.031348 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.031127 2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:50.031348 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.031136 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlpx2\" (UniqueName: \"kubernetes.io/projected/1e373d67-33bb-4271-8d89-0f4f45c612e7-kube-api-access-xlpx2\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:50.031348 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.031144 2557 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-config-volume\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:50.031348 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.031153 2557 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-cluster-tls-config\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:50.031348 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.031161 2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-main-tls\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:50.031348 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.031170 2557 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1e373d67-33bb-4271-8d89-0f4f45c612e7-alertmanager-main-db\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:50.031348 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.031180 2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:50.031348 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.031189 2557 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1e373d67-33bb-4271-8d89-0f4f45c612e7-tls-assets\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:50.031348 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.031197 2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:50.031348 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.031205 2557 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1e373d67-33bb-4271-8d89-0f4f45c612e7-web-config\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:50.120217 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.120185 2557 generic.go:356] "Generic (PLEG): container finished" podID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerID="b182e3f7f38f26ac9bfab64c49c99272bc3aaf897d819a3b714900eff7611f9b" exitCode=0 Jul 06 19:01:50.120402 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.120269 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.120402 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.120267 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e373d67-33bb-4271-8d89-0f4f45c612e7","Type":"ContainerDied","Data":"b182e3f7f38f26ac9bfab64c49c99272bc3aaf897d819a3b714900eff7611f9b"} Jul 06 19:01:50.120402 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.120385 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e373d67-33bb-4271-8d89-0f4f45c612e7","Type":"ContainerDied","Data":"f11217faa410e7956ea9850425109372c651429508776246128fbeca312f0050"} Jul 06 19:01:50.120599 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.120448 2557 scope.go:117] "RemoveContainer" containerID="fb7e991159986380b7b5d9f0a536eb216f60fe47f50627b6c53a588a65fa6072" Jul 06 19:01:50.128405 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.128383 2557 scope.go:117] "RemoveContainer" containerID="31c676b9bd07363f676e22a42fc54b774f939f4a398d489ebd44ffe315196334" Jul 06 19:01:50.136561 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.136545 2557 scope.go:117] "RemoveContainer" containerID="96cf2e67ee051bd7a24fbce22b142877aa2de6346b28de9d6a66f8dd20ee9f9a" Jul 06 19:01:50.143620 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.143604 2557 scope.go:117] "RemoveContainer" containerID="b182e3f7f38f26ac9bfab64c49c99272bc3aaf897d819a3b714900eff7611f9b" Jul 06 19:01:50.151366 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.151347 2557 scope.go:117] "RemoveContainer" containerID="9493ee6e84a3b6ba955245b86ae4a09dfa373008738176f29f0007aa559b352a" Jul 06 19:01:50.157053 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.157028 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:01:50.159580 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.159414 2557 scope.go:117] "RemoveContainer" containerID="df880413895d798542d49994722d567bc4740784029e22c2f07971941e870a0d" Jul 06 19:01:50.161179 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.161160 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:01:50.166923 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.166904 2557 scope.go:117] "RemoveContainer" containerID="5fd3f8e759dd991da4f49636a958d12d689b2cc46e181f75f465af7ad9b56eac" Jul 06 19:01:50.174280 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.174259 2557 scope.go:117] "RemoveContainer" containerID="fb7e991159986380b7b5d9f0a536eb216f60fe47f50627b6c53a588a65fa6072" Jul 06 19:01:50.174600 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:01:50.174581 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb7e991159986380b7b5d9f0a536eb216f60fe47f50627b6c53a588a65fa6072\": container with ID starting with fb7e991159986380b7b5d9f0a536eb216f60fe47f50627b6c53a588a65fa6072 not found: ID does not exist" containerID="fb7e991159986380b7b5d9f0a536eb216f60fe47f50627b6c53a588a65fa6072" Jul 06 19:01:50.174659 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.174609 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb7e991159986380b7b5d9f0a536eb216f60fe47f50627b6c53a588a65fa6072"} err="failed to get container status \"fb7e991159986380b7b5d9f0a536eb216f60fe47f50627b6c53a588a65fa6072\": rpc error: code = NotFound desc = could not find container \"fb7e991159986380b7b5d9f0a536eb216f60fe47f50627b6c53a588a65fa6072\": container with ID starting with fb7e991159986380b7b5d9f0a536eb216f60fe47f50627b6c53a588a65fa6072 not found: ID does not exist" Jul 06 19:01:50.174659 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.174628 2557 scope.go:117] "RemoveContainer" containerID="31c676b9bd07363f676e22a42fc54b774f939f4a398d489ebd44ffe315196334" Jul 06 19:01:50.174862 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:01:50.174844 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31c676b9bd07363f676e22a42fc54b774f939f4a398d489ebd44ffe315196334\": container with ID starting with 31c676b9bd07363f676e22a42fc54b774f939f4a398d489ebd44ffe315196334 not found: ID does not exist" containerID="31c676b9bd07363f676e22a42fc54b774f939f4a398d489ebd44ffe315196334" Jul 06 19:01:50.174900 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.174870 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31c676b9bd07363f676e22a42fc54b774f939f4a398d489ebd44ffe315196334"} err="failed to get container status \"31c676b9bd07363f676e22a42fc54b774f939f4a398d489ebd44ffe315196334\": rpc error: code = NotFound desc = could not find container \"31c676b9bd07363f676e22a42fc54b774f939f4a398d489ebd44ffe315196334\": container with ID starting with 31c676b9bd07363f676e22a42fc54b774f939f4a398d489ebd44ffe315196334 not found: ID does not exist" Jul 06 19:01:50.174900 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.174888 2557 scope.go:117] "RemoveContainer" containerID="96cf2e67ee051bd7a24fbce22b142877aa2de6346b28de9d6a66f8dd20ee9f9a" Jul 06 19:01:50.175148 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:01:50.175132 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96cf2e67ee051bd7a24fbce22b142877aa2de6346b28de9d6a66f8dd20ee9f9a\": container with ID starting with 96cf2e67ee051bd7a24fbce22b142877aa2de6346b28de9d6a66f8dd20ee9f9a not found: ID does not exist" containerID="96cf2e67ee051bd7a24fbce22b142877aa2de6346b28de9d6a66f8dd20ee9f9a" Jul 06 19:01:50.175206 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.175151 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96cf2e67ee051bd7a24fbce22b142877aa2de6346b28de9d6a66f8dd20ee9f9a"} err="failed to get container status \"96cf2e67ee051bd7a24fbce22b142877aa2de6346b28de9d6a66f8dd20ee9f9a\": rpc error: code = NotFound desc = could not find container \"96cf2e67ee051bd7a24fbce22b142877aa2de6346b28de9d6a66f8dd20ee9f9a\": container with ID starting with 96cf2e67ee051bd7a24fbce22b142877aa2de6346b28de9d6a66f8dd20ee9f9a not found: ID does not exist" Jul 06 19:01:50.175206 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.175164 2557 scope.go:117] "RemoveContainer" containerID="b182e3f7f38f26ac9bfab64c49c99272bc3aaf897d819a3b714900eff7611f9b" Jul 06 19:01:50.175371 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:01:50.175353 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b182e3f7f38f26ac9bfab64c49c99272bc3aaf897d819a3b714900eff7611f9b\": container with ID starting with b182e3f7f38f26ac9bfab64c49c99272bc3aaf897d819a3b714900eff7611f9b not found: ID does not exist" containerID="b182e3f7f38f26ac9bfab64c49c99272bc3aaf897d819a3b714900eff7611f9b" Jul 06 19:01:50.175413 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.175374 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b182e3f7f38f26ac9bfab64c49c99272bc3aaf897d819a3b714900eff7611f9b"} err="failed to get container status \"b182e3f7f38f26ac9bfab64c49c99272bc3aaf897d819a3b714900eff7611f9b\": rpc error: code = NotFound desc = could not find container \"b182e3f7f38f26ac9bfab64c49c99272bc3aaf897d819a3b714900eff7611f9b\": container with ID starting with b182e3f7f38f26ac9bfab64c49c99272bc3aaf897d819a3b714900eff7611f9b not found: ID does not exist" Jul 06 19:01:50.175413 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.175387 2557 scope.go:117] "RemoveContainer" containerID="9493ee6e84a3b6ba955245b86ae4a09dfa373008738176f29f0007aa559b352a" Jul 06 19:01:50.175622 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:01:50.175606 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9493ee6e84a3b6ba955245b86ae4a09dfa373008738176f29f0007aa559b352a\": container with ID starting with 9493ee6e84a3b6ba955245b86ae4a09dfa373008738176f29f0007aa559b352a not found: ID does not exist" containerID="9493ee6e84a3b6ba955245b86ae4a09dfa373008738176f29f0007aa559b352a" Jul 06 19:01:50.175660 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.175629 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9493ee6e84a3b6ba955245b86ae4a09dfa373008738176f29f0007aa559b352a"} err="failed to get container status \"9493ee6e84a3b6ba955245b86ae4a09dfa373008738176f29f0007aa559b352a\": rpc error: code = NotFound desc = could not find container \"9493ee6e84a3b6ba955245b86ae4a09dfa373008738176f29f0007aa559b352a\": container with ID starting with 9493ee6e84a3b6ba955245b86ae4a09dfa373008738176f29f0007aa559b352a not found: ID does not exist" Jul 06 19:01:50.175660 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.175643 2557 scope.go:117] "RemoveContainer" containerID="df880413895d798542d49994722d567bc4740784029e22c2f07971941e870a0d" Jul 06 19:01:50.175865 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:01:50.175851 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df880413895d798542d49994722d567bc4740784029e22c2f07971941e870a0d\": container with ID starting with df880413895d798542d49994722d567bc4740784029e22c2f07971941e870a0d not found: ID does not exist" containerID="df880413895d798542d49994722d567bc4740784029e22c2f07971941e870a0d" Jul 06 19:01:50.175908 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.175867 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df880413895d798542d49994722d567bc4740784029e22c2f07971941e870a0d"} err="failed to get container status \"df880413895d798542d49994722d567bc4740784029e22c2f07971941e870a0d\": rpc error: code = NotFound desc = could not find container \"df880413895d798542d49994722d567bc4740784029e22c2f07971941e870a0d\": container with ID starting with df880413895d798542d49994722d567bc4740784029e22c2f07971941e870a0d not found: ID does not exist" Jul 06 19:01:50.175908 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.175879 2557 scope.go:117] "RemoveContainer" containerID="5fd3f8e759dd991da4f49636a958d12d689b2cc46e181f75f465af7ad9b56eac" Jul 06 19:01:50.176076 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:01:50.176058 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fd3f8e759dd991da4f49636a958d12d689b2cc46e181f75f465af7ad9b56eac\": container with ID starting with 5fd3f8e759dd991da4f49636a958d12d689b2cc46e181f75f465af7ad9b56eac not found: ID does not exist" containerID="5fd3f8e759dd991da4f49636a958d12d689b2cc46e181f75f465af7ad9b56eac" Jul 06 19:01:50.176110 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.176081 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fd3f8e759dd991da4f49636a958d12d689b2cc46e181f75f465af7ad9b56eac"} err="failed to get container status \"5fd3f8e759dd991da4f49636a958d12d689b2cc46e181f75f465af7ad9b56eac\": rpc error: code = NotFound desc = could not find container \"5fd3f8e759dd991da4f49636a958d12d689b2cc46e181f75f465af7ad9b56eac\": container with ID starting with 5fd3f8e759dd991da4f49636a958d12d689b2cc46e181f75f465af7ad9b56eac not found: ID does not exist" Jul 06 19:01:50.196615 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.196533 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" path="/var/lib/kubelet/pods/1e373d67-33bb-4271-8d89-0f4f45c612e7/volumes" Jul 06 19:01:50.211477 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211449 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:01:50.211791 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211778 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad18adb3-6395-4684-9017-f72ec4891cdc" containerName="console" Jul 06 19:01:50.211835 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211793 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad18adb3-6395-4684-9017-f72ec4891cdc" containerName="console" Jul 06 19:01:50.211835 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211807 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="init-config-reloader" Jul 06 19:01:50.211835 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211815 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="init-config-reloader" Jul 06 19:01:50.211835 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211827 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c125efd-4ffd-4d90-ac8a-ea2133b24143" containerName="console" Jul 06 19:01:50.211835 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211833 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c125efd-4ffd-4d90-ac8a-ea2133b24143" containerName="console" Jul 06 19:01:50.211971 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211843 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="kube-rbac-proxy-metric" Jul 06 19:01:50.211971 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211849 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="kube-rbac-proxy-metric" Jul 06 19:01:50.211971 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211857 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="kube-rbac-proxy" Jul 06 19:01:50.211971 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211863 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="kube-rbac-proxy" Jul 06 19:01:50.211971 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211872 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="kube-rbac-proxy-web" Jul 06 19:01:50.211971 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211877 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="kube-rbac-proxy-web" Jul 06 19:01:50.211971 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211884 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="alertmanager" Jul 06 19:01:50.211971 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211891 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="alertmanager" Jul 06 19:01:50.211971 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211902 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="config-reloader" Jul 06 19:01:50.211971 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211910 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="config-reloader" Jul 06 19:01:50.211971 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211917 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="prom-label-proxy" Jul 06 19:01:50.211971 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211924 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="prom-label-proxy" Jul 06 19:01:50.211971 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211968 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c125efd-4ffd-4d90-ac8a-ea2133b24143" containerName="console" Jul 06 19:01:50.212398 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211979 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="prom-label-proxy" Jul 06 19:01:50.212398 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211989 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="config-reloader" Jul 06 19:01:50.212398 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.211995 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="kube-rbac-proxy-web" Jul 06 19:01:50.212398 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.212000 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="alertmanager" Jul 06 19:01:50.212398 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.212008 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="kube-rbac-proxy-metric" Jul 06 19:01:50.212398 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.212014 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e373d67-33bb-4271-8d89-0f4f45c612e7" containerName="kube-rbac-proxy" Jul 06 19:01:50.212398 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.212019 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad18adb3-6395-4684-9017-f72ec4891cdc" containerName="console" Jul 06 19:01:50.215762 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.215746 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.220834 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.220811 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 19:01:50.220959 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.220812 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 19:01:50.220959 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.220924 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 19:01:50.221523 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.221504 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 19:01:50.221644 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.221601 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 19:01:50.221743 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.221663 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fnh2m\"" Jul 06 19:01:50.221875 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.221862 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 19:01:50.223749 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.223730 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 19:01:50.224268 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.224250 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 19:01:50.230507 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.230489 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 19:01:50.268298 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.268258 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:01:50.333890 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.333850 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvf8s\" (UniqueName: \"kubernetes.io/projected/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-kube-api-access-rvf8s\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.333890 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.333894 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.334132 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.333918 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.334132 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.333944 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.334132 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.333973 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-config-volume\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.334132 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.334002 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.334132 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.334031 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.334132 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.334056 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.334132 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.334116 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-config-out\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.334390 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.334148 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.334390 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.334184 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-web-config\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.334390 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.334208 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.334390 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.334245 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.435592 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.435549 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvf8s\" (UniqueName: \"kubernetes.io/projected/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-kube-api-access-rvf8s\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.435743 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.435599 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.435743 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.435625 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.435743 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.435652 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.435743 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.435682 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-config-volume\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.435743 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.435711 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.435743 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.435740 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.436029 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.435765 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.436029 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.435813 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-config-out\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.436029 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.435846 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.436029 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.435878 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-web-config\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.436029 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.435898 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.436029 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.435919 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.437241 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.437215 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.437395 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.437263 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.438722 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.438692 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.438848 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.438818 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.439108 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.439084 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-config-volume\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.439170 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.439124 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.439227 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.439171 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.439637 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.439612 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.441969 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.440388 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.441969 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.441745 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-web-config\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.441969 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.441752 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.443723 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.443666 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-config-out\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.447943 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.447888 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvf8s\" (UniqueName: \"kubernetes.io/projected/4de5ee22-0455-4eb2-81ad-9f8e5e686b14-kube-api-access-rvf8s\") pod \"alertmanager-main-0\" (UID: \"4de5ee22-0455-4eb2-81ad-9f8e5e686b14\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.526262 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.526221 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:01:50.678152 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:50.678129 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:01:50.680936 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:01:50.680911 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4de5ee22_0455_4eb2_81ad_9f8e5e686b14.slice/crio-f899903e1065f665f37233b23fa84ea18b51f6be228281b7b4cbc7bc460cb2ea WatchSource:0}: Error finding container f899903e1065f665f37233b23fa84ea18b51f6be228281b7b4cbc7bc460cb2ea: Status 404 returned error can't find the container with id f899903e1065f665f37233b23fa84ea18b51f6be228281b7b4cbc7bc460cb2ea Jul 06 19:01:51.125973 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:51.125940 2557 generic.go:356] "Generic (PLEG): container finished" podID="4de5ee22-0455-4eb2-81ad-9f8e5e686b14" containerID="e587cc6d121f6d4b3640d05754b5f4bfd559076c65cda4683f54f3159b5c7503" exitCode=0 Jul 06 19:01:51.126419 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:51.126024 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4de5ee22-0455-4eb2-81ad-9f8e5e686b14","Type":"ContainerDied","Data":"e587cc6d121f6d4b3640d05754b5f4bfd559076c65cda4683f54f3159b5c7503"} Jul 06 19:01:51.126419 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:51.126099 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4de5ee22-0455-4eb2-81ad-9f8e5e686b14","Type":"ContainerStarted","Data":"f899903e1065f665f37233b23fa84ea18b51f6be228281b7b4cbc7bc460cb2ea"} Jul 06 19:01:52.134226 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:52.134194 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4de5ee22-0455-4eb2-81ad-9f8e5e686b14","Type":"ContainerStarted","Data":"a95999e24abeebbb16b84fa8ed68154dc060e837cd354e802a3195e6a9b6ec87"} Jul 06 19:01:52.134621 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:52.134257 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4de5ee22-0455-4eb2-81ad-9f8e5e686b14","Type":"ContainerStarted","Data":"9ca957759f75e4bedc76e79ab69f8da97c43252f64a6c43ae0dead1a0f4fe740"} Jul 06 19:01:52.134621 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:52.134283 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4de5ee22-0455-4eb2-81ad-9f8e5e686b14","Type":"ContainerStarted","Data":"85c8c78fd3fdab156beeb2d86e08b9242f6a6c99cdd0d1d058f82fbeca35a50e"} Jul 06 19:01:52.134621 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:52.134309 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4de5ee22-0455-4eb2-81ad-9f8e5e686b14","Type":"ContainerStarted","Data":"9ab1315016beecdc76e40a2bbb166a12a3891f4971ff89be0e0b31a4f49d4cee"} Jul 06 19:01:52.134621 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:52.134332 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4de5ee22-0455-4eb2-81ad-9f8e5e686b14","Type":"ContainerStarted","Data":"5380e29bb88059020021cbe962f67cd8c77c2cf74f4e276dd839b1c6c0034734"} Jul 06 19:01:52.134621 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:52.134357 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4de5ee22-0455-4eb2-81ad-9f8e5e686b14","Type":"ContainerStarted","Data":"3b1ea2c4337b09631a8f801d6e5b760f0f39eeb46bce24cc518ff4c77ea0a948"} Jul 06 19:01:52.218783 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:52.218727 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.200483089 podStartE2EDuration="2.218710819s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.22773ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:01:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:01:52.217370239 +0000 UTC m=+152.578534031" watchObservedRunningTime="2026-07-06 19:01:52.218710819 +0000 UTC m=+152.579874610" Jul 06 19:01:52.999085 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:52.999052 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:01:52.999401 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:52.999366 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="prometheus" containerID="cri-o://31c8e5d8b9305f32bb0170c1be1467bc4fbd1690c38065ae9f95ecd8fe2f5ac5" gracePeriod=600 Jul 06 19:01:52.999638 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:52.999378 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="kube-rbac-proxy" containerID="cri-o://87c3175e1027d269c648925eeed63f214d70cce9936916da3e35035cb63937cd" gracePeriod=600 Jul 06 19:01:52.999745 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:52.999392 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="thanos-sidecar" containerID="cri-o://f1189e098046c182b337e3ef5cf5b92623e18c3cbdd896ff6e56618d3871df80" gracePeriod=600 Jul 06 19:01:52.999820 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:52.999411 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="kube-rbac-proxy-thanos" containerID="cri-o://475a207ac7217978cb49c29aae51a0d409d3bff74ba4907c88c0c40b083632ce" gracePeriod=600 Jul 06 19:01:52.999918 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:52.999419 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="config-reloader" containerID="cri-o://8a73ea8ad43a4ccd68d525bb1f44087219a04505e017a4d02f1bf05bf6cdd887" gracePeriod=600 Jul 06 19:01:53.000024 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:52.999418 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="kube-rbac-proxy-web" containerID="cri-o://0c75f3d09ff3a894bc63d573e3959bc2763099a84935f233fcf2dd0f55036567" gracePeriod=600 Jul 06 19:01:53.142118 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.142082 2557 generic.go:356] "Generic (PLEG): container finished" podID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerID="475a207ac7217978cb49c29aae51a0d409d3bff74ba4907c88c0c40b083632ce" exitCode=0 Jul 06 19:01:53.142118 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.142115 2557 generic.go:356] "Generic (PLEG): container finished" podID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerID="87c3175e1027d269c648925eeed63f214d70cce9936916da3e35035cb63937cd" exitCode=0 Jul 06 19:01:53.142118 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.142125 2557 generic.go:356] "Generic (PLEG): container finished" podID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerID="0c75f3d09ff3a894bc63d573e3959bc2763099a84935f233fcf2dd0f55036567" exitCode=0 Jul 06 19:01:53.142632 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.142135 2557 generic.go:356] "Generic (PLEG): container finished" podID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerID="f1189e098046c182b337e3ef5cf5b92623e18c3cbdd896ff6e56618d3871df80" exitCode=0 Jul 06 19:01:53.142632 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.142145 2557 generic.go:356] "Generic (PLEG): container finished" podID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerID="8a73ea8ad43a4ccd68d525bb1f44087219a04505e017a4d02f1bf05bf6cdd887" exitCode=0 Jul 06 19:01:53.142632 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.142153 2557 generic.go:356] "Generic (PLEG): container finished" podID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerID="31c8e5d8b9305f32bb0170c1be1467bc4fbd1690c38065ae9f95ecd8fe2f5ac5" exitCode=0 Jul 06 19:01:53.142632 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.142146 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"36719a7e-f0d4-4c37-b625-d9e5bd3827d4","Type":"ContainerDied","Data":"475a207ac7217978cb49c29aae51a0d409d3bff74ba4907c88c0c40b083632ce"} Jul 06 19:01:53.142632 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.142228 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"36719a7e-f0d4-4c37-b625-d9e5bd3827d4","Type":"ContainerDied","Data":"87c3175e1027d269c648925eeed63f214d70cce9936916da3e35035cb63937cd"} Jul 06 19:01:53.142632 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.142277 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"36719a7e-f0d4-4c37-b625-d9e5bd3827d4","Type":"ContainerDied","Data":"0c75f3d09ff3a894bc63d573e3959bc2763099a84935f233fcf2dd0f55036567"} Jul 06 19:01:53.142632 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.142317 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"36719a7e-f0d4-4c37-b625-d9e5bd3827d4","Type":"ContainerDied","Data":"f1189e098046c182b337e3ef5cf5b92623e18c3cbdd896ff6e56618d3871df80"} Jul 06 19:01:53.142632 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.142358 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"36719a7e-f0d4-4c37-b625-d9e5bd3827d4","Type":"ContainerDied","Data":"8a73ea8ad43a4ccd68d525bb1f44087219a04505e017a4d02f1bf05bf6cdd887"} Jul 06 19:01:53.142632 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.142396 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"36719a7e-f0d4-4c37-b625-d9e5bd3827d4","Type":"ContainerDied","Data":"31c8e5d8b9305f32bb0170c1be1467bc4fbd1690c38065ae9f95ecd8fe2f5ac5"} Jul 06 19:01:53.247791 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.247759 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:53.362166 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362068 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-kubelet-serving-ca-bundle\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362166 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362113 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-web-config\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362166 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362148 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-metrics-client-certs\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362451 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362177 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362451 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362208 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n79dg\" (UniqueName: \"kubernetes.io/projected/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-kube-api-access-n79dg\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362451 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362235 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-kube-rbac-proxy\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362451 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362251 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-serving-certs-ca-bundle\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362451 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362280 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-thanos-prometheus-http-client-file\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362451 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362305 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-tls-assets\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362451 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362342 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-config-out\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362451 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362397 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-trusted-ca-bundle\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362451 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362425 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-1\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362928 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362454 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:01:53.362928 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362477 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-0\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362928 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362504 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-tls\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362928 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362529 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-metrics-client-ca\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362928 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362572 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-grpc-tls\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362928 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362605 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-config\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.362928 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.362804 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:01:53.363279 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.363251 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:01:53.363399 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.363382 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:01:53.363681 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.363656 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:01:53.365707 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.364489 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-db\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.365707 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.364569 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.365707 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.364606 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-2\") pod \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\" (UID: \"36719a7e-f0d4-4c37-b625-d9e5bd3827d4\") " Jul 06 19:01:53.365707 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.364910 2557 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.365707 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.364932 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.365707 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.364949 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.365707 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.364964 2557 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-metrics-client-ca\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.365707 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.364979 2557 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.365707 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.365363 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-config-out" (OuterVolumeSpecName: "config-out") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:01:53.365707 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.365666 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:01:53.366344 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.366320 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:01:53.366564 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.366524 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:01:53.366794 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.366739 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:53.366988 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.366917 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-kube-api-access-n79dg" (OuterVolumeSpecName: "kube-api-access-n79dg") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "kube-api-access-n79dg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:01:53.367158 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.367119 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:53.367354 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.367330 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:53.367735 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.367627 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:53.367735 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.367699 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:53.367888 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.367864 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:53.368309 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.368286 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:01:53.368764 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.368732 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-config" (OuterVolumeSpecName: "config") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:53.369029 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.369001 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:53.376918 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.376890 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-web-config" (OuterVolumeSpecName: "web-config") pod "36719a7e-f0d4-4c37-b625-d9e5bd3827d4" (UID: "36719a7e-f0d4-4c37-b625-d9e5bd3827d4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:01:53.420961 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.420929 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b4c9dc875-lf457"] Jul 06 19:01:53.421278 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421265 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="kube-rbac-proxy-thanos" Jul 06 19:01:53.421322 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421279 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="kube-rbac-proxy-thanos" Jul 06 19:01:53.421322 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421287 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="init-config-reloader" Jul 06 19:01:53.421322 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421293 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="init-config-reloader" Jul 06 19:01:53.421322 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421303 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="kube-rbac-proxy" Jul 06 19:01:53.421322 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421309 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="kube-rbac-proxy" Jul 06 19:01:53.421322 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421315 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="thanos-sidecar" Jul 06 19:01:53.421322 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421320 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="thanos-sidecar" Jul 06 19:01:53.421544 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421337 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="prometheus" Jul 06 19:01:53.421544 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421342 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="prometheus" Jul 06 19:01:53.421544 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421349 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="kube-rbac-proxy-web" Jul 06 19:01:53.421544 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421354 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="kube-rbac-proxy-web" Jul 06 19:01:53.421544 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421361 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="config-reloader" Jul 06 19:01:53.421544 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421366 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="config-reloader" Jul 06 19:01:53.421544 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421412 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="prometheus" Jul 06 19:01:53.421544 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421422 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="kube-rbac-proxy-web" Jul 06 19:01:53.421544 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421428 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="thanos-sidecar" Jul 06 19:01:53.421544 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421448 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="kube-rbac-proxy" Jul 06 19:01:53.421544 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421453 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="kube-rbac-proxy-thanos" Jul 06 19:01:53.421544 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.421458 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" containerName="config-reloader" Jul 06 19:01:53.424200 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.424182 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.430012 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.429983 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 19:01:53.430298 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.430279 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 19:01:53.430478 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.430306 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 19:01:53.430590 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.430569 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 19:01:53.430590 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.430570 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vzl4z\"" Jul 06 19:01:53.430754 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.430652 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 19:01:53.444679 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.444653 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 19:01:53.450326 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.450297 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b4c9dc875-lf457"] Jul 06 19:01:53.465992 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.465962 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dec7f68-ed04-49be-a571-68a52ecfa052-console-serving-cert\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.466169 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.465997 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dec7f68-ed04-49be-a571-68a52ecfa052-console-oauth-config\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.466169 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466022 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-oauth-serving-cert\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.466169 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466045 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-service-ca\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.466169 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466152 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mkbf\" (UniqueName: \"kubernetes.io/projected/6dec7f68-ed04-49be-a571-68a52ecfa052-kube-api-access-9mkbf\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.466384 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466225 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-trusted-ca-bundle\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.466384 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466258 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-console-config\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.466384 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466355 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.466384 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466373 2557 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.466601 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466389 2557 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-grpc-tls\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.466601 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466403 2557 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-config\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.466601 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466418 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-db\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.466601 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466452 2557 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.466601 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466469 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.466601 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466484 2557 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-web-config\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.466601 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466498 2557 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-metrics-client-certs\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.466601 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466514 2557 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.466601 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466530 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n79dg\" (UniqueName: \"kubernetes.io/projected/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-kube-api-access-n79dg\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.466601 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466545 2557 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-secret-kube-rbac-proxy\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.466601 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466560 2557 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.466601 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466575 2557 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-tls-assets\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.466601 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.466589 2557 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/36719a7e-f0d4-4c37-b625-d9e5bd3827d4-config-out\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:01:53.534544 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.534507 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:53.567005 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.566969 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mkbf\" (UniqueName: \"kubernetes.io/projected/6dec7f68-ed04-49be-a571-68a52ecfa052-kube-api-access-9mkbf\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.567150 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.567018 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-trusted-ca-bundle\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.567150 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.567046 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-console-config\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.567150 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.567081 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dec7f68-ed04-49be-a571-68a52ecfa052-console-serving-cert\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.567150 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.567106 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dec7f68-ed04-49be-a571-68a52ecfa052-console-oauth-config\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.567150 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.567141 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-oauth-serving-cert\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.567410 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.567180 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-service-ca\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.567951 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.567903 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-oauth-serving-cert\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.567951 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.567928 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-console-config\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.568093 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.568025 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-service-ca\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.568291 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.568272 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-trusted-ca-bundle\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.569604 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.569582 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dec7f68-ed04-49be-a571-68a52ecfa052-console-oauth-config\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.569779 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.569764 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dec7f68-ed04-49be-a571-68a52ecfa052-console-serving-cert\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.588461 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.588406 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mkbf\" (UniqueName: \"kubernetes.io/projected/6dec7f68-ed04-49be-a571-68a52ecfa052-kube-api-access-9mkbf\") pod \"console-7b4c9dc875-lf457\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.644379 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.644279 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:53.734042 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.734005 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:01:53.817120 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.817067 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:53.877391 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:53.877359 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b4c9dc875-lf457"] Jul 06 19:01:53.880280 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:01:53.880249 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dec7f68_ed04_49be_a571_68a52ecfa052.slice/crio-479a5e29494192ae8a9654a289f780cb2f5f7fe7016db41f2ad7c2e8c61fe27d WatchSource:0}: Error finding container 479a5e29494192ae8a9654a289f780cb2f5f7fe7016db41f2ad7c2e8c61fe27d: Status 404 returned error can't find the container with id 479a5e29494192ae8a9654a289f780cb2f5f7fe7016db41f2ad7c2e8c61fe27d Jul 06 19:01:54.150293 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.150192 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"36719a7e-f0d4-4c37-b625-d9e5bd3827d4","Type":"ContainerDied","Data":"7fd9c423bbfa4e18b5ff00f56480deb93e6921380faa8a8a8ea2f928a3a5da04"} Jul 06 19:01:54.150293 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.150250 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.150293 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.150282 2557 scope.go:117] "RemoveContainer" containerID="475a207ac7217978cb49c29aae51a0d409d3bff74ba4907c88c0c40b083632ce" Jul 06 19:01:54.151965 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.151943 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b4c9dc875-lf457" event={"ID":"6dec7f68-ed04-49be-a571-68a52ecfa052","Type":"ContainerStarted","Data":"50585cb194b15b330507a7f0e303f5a1bb6d2fe1c8712dc79d251b6f5d5a5c5e"} Jul 06 19:01:54.152077 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.151979 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b4c9dc875-lf457" event={"ID":"6dec7f68-ed04-49be-a571-68a52ecfa052","Type":"ContainerStarted","Data":"479a5e29494192ae8a9654a289f780cb2f5f7fe7016db41f2ad7c2e8c61fe27d"} Jul 06 19:01:54.159704 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.159689 2557 scope.go:117] "RemoveContainer" containerID="87c3175e1027d269c648925eeed63f214d70cce9936916da3e35035cb63937cd" Jul 06 19:01:54.167678 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.167662 2557 scope.go:117] "RemoveContainer" containerID="0c75f3d09ff3a894bc63d573e3959bc2763099a84935f233fcf2dd0f55036567" Jul 06 19:01:54.174558 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.174536 2557 scope.go:117] "RemoveContainer" containerID="f1189e098046c182b337e3ef5cf5b92623e18c3cbdd896ff6e56618d3871df80" Jul 06 19:01:54.181107 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.181091 2557 scope.go:117] "RemoveContainer" containerID="8a73ea8ad43a4ccd68d525bb1f44087219a04505e017a4d02f1bf05bf6cdd887" Jul 06 19:01:54.187811 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.187772 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b4c9dc875-lf457" podStartSLOduration=1.187759084 podStartE2EDuration="1.187759084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:01:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:01:54.185829968 +0000 UTC m=+154.546993771" watchObservedRunningTime="2026-07-06 19:01:54.187759084 +0000 UTC m=+154.548922873" Jul 06 19:01:54.188283 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.188265 2557 scope.go:117] "RemoveContainer" containerID="31c8e5d8b9305f32bb0170c1be1467bc4fbd1690c38065ae9f95ecd8fe2f5ac5" Jul 06 19:01:54.196232 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.196217 2557 scope.go:117] "RemoveContainer" containerID="f4b543433a6c9b0e42c702595e979b49efbb7e1a2490249ac9b363736d00c8b8" Jul 06 19:01:54.210543 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.210520 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:01:54.216988 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.216965 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:01:54.225864 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.225829 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:54.263415 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.263383 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:01:54.267616 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.267598 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.276728 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.276706 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 19:01:54.276850 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.276701 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 19:01:54.279518 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.279500 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 19:01:54.279635 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.279566 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 19:01:54.279739 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.279641 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3ek2knif8baik\"" Jul 06 19:01:54.280118 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.280106 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 19:01:54.280162 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.280119 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 19:01:54.280386 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.280371 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 19:01:54.280643 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.280631 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 19:01:54.281309 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.281294 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 19:01:54.282070 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.282048 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 19:01:54.285218 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.285203 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 19:01:54.287946 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.287929 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 19:01:54.288285 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.288272 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-45pqd\"" Jul 06 19:01:54.288285 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.288281 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 19:01:54.297466 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.296363 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 19:01:54.302803 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.302783 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:01:54.366373 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.366335 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:54.374391 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374361 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.374568 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374409 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-web-config\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.374568 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374461 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.374568 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374488 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.374568 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374538 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.374781 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374568 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-config\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.374781 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374604 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.374781 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374636 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.374781 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374666 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/78fdb525-75bb-444c-a0c2-435dcac110e8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.374781 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374692 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.374781 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374715 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/78fdb525-75bb-444c-a0c2-435dcac110e8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.374781 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374756 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.375051 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374779 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjcg6\" (UniqueName: \"kubernetes.io/projected/78fdb525-75bb-444c-a0c2-435dcac110e8-kube-api-access-sjcg6\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.375051 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374848 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/78fdb525-75bb-444c-a0c2-435dcac110e8-config-out\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.375051 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374873 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.375051 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374901 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.375051 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374924 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.375051 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.374986 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.375051 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.375028 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.375051 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.375050 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.476305 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476213 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.476305 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476256 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.476305 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476282 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/78fdb525-75bb-444c-a0c2-435dcac110e8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.476566 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476464 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.476566 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476493 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/78fdb525-75bb-444c-a0c2-435dcac110e8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.476566 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476540 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.476698 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476568 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjcg6\" (UniqueName: \"kubernetes.io/projected/78fdb525-75bb-444c-a0c2-435dcac110e8-kube-api-access-sjcg6\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.476698 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476615 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/78fdb525-75bb-444c-a0c2-435dcac110e8-config-out\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.476698 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476640 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.476698 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476669 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.476698 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476693 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.477017 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476730 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.477017 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476769 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.477017 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476794 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.477017 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476827 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.477017 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476872 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-web-config\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.477017 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476900 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.477017 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476905 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/78fdb525-75bb-444c-a0c2-435dcac110e8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.477017 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476924 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.477017 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.476986 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.477495 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.477034 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-config\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.477495 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.477038 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.477495 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.477106 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.477652 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.477606 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.477867 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.477838 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.478358 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.478338 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.478512 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.478404 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.479496 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.479468 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.479715 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.479675 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.479787 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.479774 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/78fdb525-75bb-444c-a0c2-435dcac110e8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.479978 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.479952 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.480413 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.480387 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-config\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.480780 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.480740 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/78fdb525-75bb-444c-a0c2-435dcac110e8-config-out\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.482173 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.482050 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.482173 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.482117 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.482328 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.482225 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.482429 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.482408 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.482539 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.482500 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/78fdb525-75bb-444c-a0c2-435dcac110e8-web-config\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.482596 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.482580 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/78fdb525-75bb-444c-a0c2-435dcac110e8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.489113 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.489093 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjcg6\" (UniqueName: \"kubernetes.io/projected/78fdb525-75bb-444c-a0c2-435dcac110e8-kube-api-access-sjcg6\") pod \"prometheus-k8s-0\" (UID: \"78fdb525-75bb-444c-a0c2-435dcac110e8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.577135 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.577089 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:01:54.734447 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:54.734405 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:01:54.736623 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:01:54.736599 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78fdb525_75bb_444c_a0c2_435dcac110e8.slice/crio-652bb936f0181ab6841f8501010d84e396914fffe50b0b3f26d9b9a3751b73b6 WatchSource:0}: Error finding container 652bb936f0181ab6841f8501010d84e396914fffe50b0b3f26d9b9a3751b73b6: Status 404 returned error can't find the container with id 652bb936f0181ab6841f8501010d84e396914fffe50b0b3f26d9b9a3751b73b6 Jul 06 19:01:55.157228 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:55.157192 2557 generic.go:356] "Generic (PLEG): container finished" podID="78fdb525-75bb-444c-a0c2-435dcac110e8" containerID="4dbef0865f04484b75b34d7d4cb6aedb94d83768c0c7754af149a43c6d2d9790" exitCode=0 Jul 06 19:01:55.157689 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:55.157279 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"78fdb525-75bb-444c-a0c2-435dcac110e8","Type":"ContainerDied","Data":"4dbef0865f04484b75b34d7d4cb6aedb94d83768c0c7754af149a43c6d2d9790"} Jul 06 19:01:55.157689 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:55.157356 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"78fdb525-75bb-444c-a0c2-435dcac110e8","Type":"ContainerStarted","Data":"652bb936f0181ab6841f8501010d84e396914fffe50b0b3f26d9b9a3751b73b6"} Jul 06 19:01:56.002262 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:56.002221 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:56.111169 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:56.111135 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:56.167109 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:56.167070 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"78fdb525-75bb-444c-a0c2-435dcac110e8","Type":"ContainerStarted","Data":"7f67627be8c5cd0be8697b144d3e0f166905491e73323124699db11dd0d58274"} Jul 06 19:01:56.167519 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:56.167157 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"78fdb525-75bb-444c-a0c2-435dcac110e8","Type":"ContainerStarted","Data":"84e4bc600bbd8c39c91a5c56bf2b6d1fa6cc332c526b3cf9e8b10196f1759833"} Jul 06 19:01:56.167519 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:56.167201 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"78fdb525-75bb-444c-a0c2-435dcac110e8","Type":"ContainerStarted","Data":"c6c69bfe52ae2fa50fd1a4e92c271dfc5ba583ab3798f4d4ac8c0eb03c02f10c"} Jul 06 19:01:56.167519 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:56.167243 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"78fdb525-75bb-444c-a0c2-435dcac110e8","Type":"ContainerStarted","Data":"9395a4de126ea69f48f44c5f3dba39ea1bc106b93931dd859b22960d9409d60a"} Jul 06 19:01:56.167519 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:56.167279 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"78fdb525-75bb-444c-a0c2-435dcac110e8","Type":"ContainerStarted","Data":"a964695e6cadf2243639389d98d09fac5f61d1de1e580b23ddc912da68a350cc"} Jul 06 19:01:56.167519 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:56.167322 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"78fdb525-75bb-444c-a0c2-435dcac110e8","Type":"ContainerStarted","Data":"a608a3350079f3dd1c79c72177a7606de28c97c16eb4bdbfc1b80243ffddde9f"} Jul 06 19:01:56.197505 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:56.197466 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36719a7e-f0d4-4c37-b625-d9e5bd3827d4" path="/var/lib/kubelet/pods/36719a7e-f0d4-4c37-b625-d9e5bd3827d4/volumes" Jul 06 19:01:56.223566 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:56.223511 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.205204995 podStartE2EDuration="2.223491505s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.28651ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:01:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:01:56.223081317 +0000 UTC m=+156.584245110" watchObservedRunningTime="2026-07-06 19:01:56.223491505 +0000 UTC m=+156.584655301" Jul 06 19:01:59.578154 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:01:59.578103 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:02:02.008508 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:02.008458 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:02.236857 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:02.236815 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:02.443825 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:02.443732 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:03.734696 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:03.734657 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:02:03.735132 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:03.734726 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:02:03.739537 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:03.739511 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:02:04.205303 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:04.205270 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:02:04.617954 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:04.617911 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:04.775076 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:04.775038 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:09.475006 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:09.474974 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:40.862949 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:40.862914 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:40.947153 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:40.947107 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:50.924656 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:50.924616 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:51.180504 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:51.180389 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:54.577602 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:54.577573 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:02:54.593659 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:54.593619 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:02:55.425805 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:55.425769 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:02:56.704520 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:56.704481 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:56.838114 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:02:56.838079 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:42.456792 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:03:42.456702 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:20.020807 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:20.020780 2557 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 19:04:52.097070 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:52.097032 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:53.455528 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:53.455478 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:56.101241 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:56.101198 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:56.208205 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:56.208159 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:58.224676 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:58.224636 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:58.355258 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:58.355225 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:58.774851 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:58.774812 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:58.949628 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:58.949590 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:59.214078 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:59.213994 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f7vb7"] Jul 06 19:04:59.216379 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:59.216363 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-f7vb7" Jul 06 19:04:59.222692 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:59.222672 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 19:04:59.223615 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:59.223596 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-84jqm\"" Jul 06 19:04:59.223705 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:59.223613 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 19:04:59.236273 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:59.236246 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f7vb7"] Jul 06 19:04:59.302885 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:59.302850 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sps4w\" (UniqueName: \"kubernetes.io/projected/02fdd29c-6c72-4229-92c1-f69132ee1706-kube-api-access-sps4w\") pod \"cert-manager-webhook-6486ffd997-f7vb7\" (UID: \"02fdd29c-6c72-4229-92c1-f69132ee1706\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7vb7" Jul 06 19:04:59.303046 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:59.302912 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02fdd29c-6c72-4229-92c1-f69132ee1706-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f7vb7\" (UID: \"02fdd29c-6c72-4229-92c1-f69132ee1706\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7vb7" Jul 06 19:04:59.404416 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:59.404368 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sps4w\" (UniqueName: \"kubernetes.io/projected/02fdd29c-6c72-4229-92c1-f69132ee1706-kube-api-access-sps4w\") pod \"cert-manager-webhook-6486ffd997-f7vb7\" (UID: \"02fdd29c-6c72-4229-92c1-f69132ee1706\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7vb7" Jul 06 19:04:59.404614 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:59.404479 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02fdd29c-6c72-4229-92c1-f69132ee1706-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f7vb7\" (UID: \"02fdd29c-6c72-4229-92c1-f69132ee1706\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7vb7" Jul 06 19:04:59.427309 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:59.427271 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sps4w\" (UniqueName: \"kubernetes.io/projected/02fdd29c-6c72-4229-92c1-f69132ee1706-kube-api-access-sps4w\") pod \"cert-manager-webhook-6486ffd997-f7vb7\" (UID: \"02fdd29c-6c72-4229-92c1-f69132ee1706\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7vb7" Jul 06 19:04:59.433196 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:59.433163 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02fdd29c-6c72-4229-92c1-f69132ee1706-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f7vb7\" (UID: \"02fdd29c-6c72-4229-92c1-f69132ee1706\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7vb7" Jul 06 19:04:59.541217 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:59.541185 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-f7vb7" Jul 06 19:04:59.679548 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:59.679524 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f7vb7"] Jul 06 19:04:59.681328 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:04:59.681296 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02fdd29c_6c72_4229_92c1_f69132ee1706.slice/crio-e9e696425761e60df6824caa228c9e2056290e9c70874856b95a211c88d3884e WatchSource:0}: Error finding container e9e696425761e60df6824caa228c9e2056290e9c70874856b95a211c88d3884e: Status 404 returned error can't find the container with id e9e696425761e60df6824caa228c9e2056290e9c70874856b95a211c88d3884e Jul 06 19:04:59.683245 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:59.683229 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:04:59.927925 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:04:59.927843 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-f7vb7" event={"ID":"02fdd29c-6c72-4229-92c1-f69132ee1706","Type":"ContainerStarted","Data":"e9e696425761e60df6824caa228c9e2056290e9c70874856b95a211c88d3884e"} Jul 06 19:05:03.949992 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:03.949955 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-f7vb7" event={"ID":"02fdd29c-6c72-4229-92c1-f69132ee1706","Type":"ContainerStarted","Data":"a413f6c5a2440ce7448f06a74af0ab45aab5d7180f576fec9e5924f33d9f9318"} Jul 06 19:05:03.950412 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:03.950081 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-f7vb7" Jul 06 19:05:09.779004 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:09.778967 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:09.897268 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:09.897232 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:09.958799 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:09.958764 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-f7vb7" Jul 06 19:05:10.005982 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:10.005923 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-f7vb7" podStartSLOduration=7.804295913 podStartE2EDuration="11.005905563s" totalImagesPullingTime="3.20160965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:05:07.892862109 +0000 UTC m=+348.254025908" watchObservedRunningTime="2026-07-06 19:05:10.005905563 +0000 UTC m=+350.367069354" Jul 06 19:05:14.761809 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:14.761708 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:14.892649 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:14.892618 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:16.051900 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:16.051860 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:16.065128 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:16.065098 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xr8lb"] Jul 06 19:05:16.071198 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:16.071179 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xr8lb" Jul 06 19:05:16.074703 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:16.074674 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 19:05:16.075590 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:16.075571 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:05:16.075721 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:16.075616 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-5djhk\"" Jul 06 19:05:16.083361 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:16.083336 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xr8lb"] Jul 06 19:05:16.154968 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:16.154932 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:16.158951 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:16.158932 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75e93154-0e22-4ee5-8a96-a0ba96b3de29-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xr8lb\" (UID: \"75e93154-0e22-4ee5-8a96-a0ba96b3de29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xr8lb" Jul 06 19:05:16.159020 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:16.159002 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fskcm\" (UniqueName: \"kubernetes.io/projected/75e93154-0e22-4ee5-8a96-a0ba96b3de29-kube-api-access-fskcm\") pod \"openshift-lws-operator-fd8ccff4c-xr8lb\" (UID: \"75e93154-0e22-4ee5-8a96-a0ba96b3de29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xr8lb" Jul 06 19:05:16.259737 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:16.259701 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75e93154-0e22-4ee5-8a96-a0ba96b3de29-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xr8lb\" (UID: \"75e93154-0e22-4ee5-8a96-a0ba96b3de29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xr8lb" Jul 06 19:05:16.259918 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:16.259815 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fskcm\" (UniqueName: \"kubernetes.io/projected/75e93154-0e22-4ee5-8a96-a0ba96b3de29-kube-api-access-fskcm\") pod \"openshift-lws-operator-fd8ccff4c-xr8lb\" (UID: \"75e93154-0e22-4ee5-8a96-a0ba96b3de29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xr8lb" Jul 06 19:05:16.260111 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:16.260092 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75e93154-0e22-4ee5-8a96-a0ba96b3de29-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xr8lb\" (UID: \"75e93154-0e22-4ee5-8a96-a0ba96b3de29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xr8lb" Jul 06 19:05:16.272570 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:16.272540 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fskcm\" (UniqueName: \"kubernetes.io/projected/75e93154-0e22-4ee5-8a96-a0ba96b3de29-kube-api-access-fskcm\") pod \"openshift-lws-operator-fd8ccff4c-xr8lb\" (UID: \"75e93154-0e22-4ee5-8a96-a0ba96b3de29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xr8lb" Jul 06 19:05:16.380751 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:16.380662 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xr8lb" Jul 06 19:05:16.515391 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:16.515366 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xr8lb"] Jul 06 19:05:16.517353 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:05:16.517326 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75e93154_0e22_4ee5_8a96_a0ba96b3de29.slice/crio-a424eeb6e690c737c8ec9f095c092e3ea95012e723f3c50e8bc3f3878c1cfb67 WatchSource:0}: Error finding container a424eeb6e690c737c8ec9f095c092e3ea95012e723f3c50e8bc3f3878c1cfb67: Status 404 returned error can't find the container with id a424eeb6e690c737c8ec9f095c092e3ea95012e723f3c50e8bc3f3878c1cfb67 Jul 06 19:05:17.007062 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:17.007018 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xr8lb" event={"ID":"75e93154-0e22-4ee5-8a96-a0ba96b3de29","Type":"ContainerStarted","Data":"a424eeb6e690c737c8ec9f095c092e3ea95012e723f3c50e8bc3f3878c1cfb67"} Jul 06 19:05:20.024990 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:20.024947 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xr8lb" event={"ID":"75e93154-0e22-4ee5-8a96-a0ba96b3de29","Type":"ContainerStarted","Data":"34fd080e749c4a70c33cac5cd9cb7e7666931deb7e3c9ff06b622a96c764f779"} Jul 06 19:05:20.044462 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:20.044384 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xr8lb" podStartSLOduration=1.287345693 podStartE2EDuration="4.044364538s" totalImagesPullingTime="2.757018845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:05:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:05:20.043184928 +0000 UTC m=+360.404348745" watchObservedRunningTime="2026-07-06 19:05:20.044364538 +0000 UTC m=+360.405528333" Jul 06 19:05:20.760771 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:20.760723 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:20.855042 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:20.855005 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:22.420080 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:22.420043 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:34.503504 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:34.503463 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:34.624242 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:34.624191 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:35.086663 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:35.086620 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:35.667031 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:35.666993 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:35.781585 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:35.781548 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:35.898085 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:35.898051 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m"] Jul 06 19:05:35.901846 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:35.901822 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:35.907466 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:35.907428 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 19:05:35.907466 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:35.907459 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 19:05:35.908039 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:35.908019 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 19:05:35.908220 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:35.908189 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 19:05:35.908308 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:35.908249 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kmtgc\"" Jul 06 19:05:35.922985 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:35.922898 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2780119-d899-424a-895e-9ef16b4c3cab-tmp\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m\" (UID: \"c2780119-d899-424a-895e-9ef16b4c3cab\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:35.922985 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:35.922969 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwd2c\" (UniqueName: \"kubernetes.io/projected/c2780119-d899-424a-895e-9ef16b4c3cab-kube-api-access-lwd2c\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m\" (UID: \"c2780119-d899-424a-895e-9ef16b4c3cab\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:35.923175 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:35.923031 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2780119-d899-424a-895e-9ef16b4c3cab-webhook-cert\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m\" (UID: \"c2780119-d899-424a-895e-9ef16b4c3cab\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:35.923175 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:35.923065 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2780119-d899-424a-895e-9ef16b4c3cab-manifests\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m\" (UID: \"c2780119-d899-424a-895e-9ef16b4c3cab\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:35.923175 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:35.923095 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2780119-d899-424a-895e-9ef16b4c3cab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m\" (UID: \"c2780119-d899-424a-895e-9ef16b4c3cab\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:35.935588 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:35.935562 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m"] Jul 06 19:05:36.023750 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.023713 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2780119-d899-424a-895e-9ef16b4c3cab-manifests\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m\" (UID: \"c2780119-d899-424a-895e-9ef16b4c3cab\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:36.023750 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.023760 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2780119-d899-424a-895e-9ef16b4c3cab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m\" (UID: \"c2780119-d899-424a-895e-9ef16b4c3cab\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:36.024025 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.023895 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2780119-d899-424a-895e-9ef16b4c3cab-tmp\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m\" (UID: \"c2780119-d899-424a-895e-9ef16b4c3cab\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:36.024025 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.024004 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwd2c\" (UniqueName: \"kubernetes.io/projected/c2780119-d899-424a-895e-9ef16b4c3cab-kube-api-access-lwd2c\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m\" (UID: \"c2780119-d899-424a-895e-9ef16b4c3cab\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:36.024131 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.024099 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2780119-d899-424a-895e-9ef16b4c3cab-webhook-cert\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m\" (UID: \"c2780119-d899-424a-895e-9ef16b4c3cab\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:36.024263 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.024233 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2780119-d899-424a-895e-9ef16b4c3cab-manifests\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m\" (UID: \"c2780119-d899-424a-895e-9ef16b4c3cab\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:36.026159 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.026135 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2780119-d899-424a-895e-9ef16b4c3cab-tmp\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m\" (UID: \"c2780119-d899-424a-895e-9ef16b4c3cab\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:36.026408 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.026385 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2780119-d899-424a-895e-9ef16b4c3cab-webhook-cert\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m\" (UID: \"c2780119-d899-424a-895e-9ef16b4c3cab\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:36.026474 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.026413 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2780119-d899-424a-895e-9ef16b4c3cab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m\" (UID: \"c2780119-d899-424a-895e-9ef16b4c3cab\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:36.035393 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.035372 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwd2c\" (UniqueName: \"kubernetes.io/projected/c2780119-d899-424a-895e-9ef16b4c3cab-kube-api-access-lwd2c\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m\" (UID: \"c2780119-d899-424a-895e-9ef16b4c3cab\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:36.107515 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.107480 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:36.208547 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.208448 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:36.212493 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.212477 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:36.305858 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.305822 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:36.350918 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.350897 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m"] Jul 06 19:05:36.353704 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:05:36.353675 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2780119_d899_424a_895e_9ef16b4c3cab.slice/crio-a8e01c6505d417ad72cad16b8dc1d37c938f972c1dd2b294f9f8515964c34315 WatchSource:0}: Error finding container a8e01c6505d417ad72cad16b8dc1d37c938f972c1dd2b294f9f8515964c34315: Status 404 returned error can't find the container with id a8e01c6505d417ad72cad16b8dc1d37c938f972c1dd2b294f9f8515964c34315 Jul 06 19:05:36.408318 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.408278 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:36.502769 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.502713 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:36.600397 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.600359 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:36.704556 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:36.704517 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:37.099969 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:37.099938 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" event={"ID":"c2780119-d899-424a-895e-9ef16b4c3cab","Type":"ContainerStarted","Data":"a8e01c6505d417ad72cad16b8dc1d37c938f972c1dd2b294f9f8515964c34315"} Jul 06 19:05:40.119269 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:40.119230 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" event={"ID":"c2780119-d899-424a-895e-9ef16b4c3cab","Type":"ContainerStarted","Data":"2ee49953129f1ec8dccb32abe1055c25cde520f11af29e201e84a1d6dd93fc45"} Jul 06 19:05:41.124786 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:41.124755 2557 generic.go:356] "Generic (PLEG): container finished" podID="c2780119-d899-424a-895e-9ef16b4c3cab" containerID="2ee49953129f1ec8dccb32abe1055c25cde520f11af29e201e84a1d6dd93fc45" exitCode=0 Jul 06 19:05:41.125176 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:41.124840 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" event={"ID":"c2780119-d899-424a-895e-9ef16b4c3cab","Type":"ContainerDied","Data":"2ee49953129f1ec8dccb32abe1055c25cde520f11af29e201e84a1d6dd93fc45"} Jul 06 19:05:42.131079 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:42.131050 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" event={"ID":"c2780119-d899-424a-895e-9ef16b4c3cab","Type":"ContainerStarted","Data":"e2449e1a86ed382152aaef0fbbf092cf5836dc7559c2b2ae4a22ea0cbf458102"} Jul 06 19:05:42.131511 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:42.131163 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:42.155246 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:42.155191 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" podStartSLOduration=2.9051156860000003 podStartE2EDuration="7.155172025s" totalImagesPullingTime="3.155280152s" totalInitContainerRuntime="1.094776187s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:05:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:05:42.154062361 +0000 UTC m=+382.515226152" watchObservedRunningTime="2026-07-06 19:05:42.155172025 +0000 UTC m=+382.516335816" Jul 06 19:05:45.331489 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:45.331422 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:45.456774 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:45.456736 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:53.138188 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:53.138160 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:05:55.594282 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:55.594242 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:55.780414 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:55.780375 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:56.471428 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:56.471378 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:57.111584 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:57.111546 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:57.486350 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:57.486258 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:57.753942 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:57.753888 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:58.401468 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:58.401406 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:59.022094 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:59.022058 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:59.424262 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:59.424159 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:59.612601 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:05:59.612562 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:06:00.374627 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:00.374589 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:06:00.546911 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:00.546876 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:06:03.305078 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:03.305043 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:06:04.346393 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.346359 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5"] Jul 06 19:06:04.357402 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.357373 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:06:04.361269 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.361248 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 19:06:04.363451 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.363412 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 19:06:04.363548 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.363472 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-brm4b\"" Jul 06 19:06:04.363548 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.363503 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 19:06:04.396928 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.396893 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5"] Jul 06 19:06:04.490048 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.490008 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b53c91-e1dc-4f9f-942c-e9e6612cc4c7-cert\") pod \"lws-controller-manager-84979c6fd9-87xf5\" (UID: \"61b53c91-e1dc-4f9f-942c-e9e6612cc4c7\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:06:04.490255 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.490075 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggpgl\" (UniqueName: \"kubernetes.io/projected/61b53c91-e1dc-4f9f-942c-e9e6612cc4c7-kube-api-access-ggpgl\") pod \"lws-controller-manager-84979c6fd9-87xf5\" (UID: \"61b53c91-e1dc-4f9f-942c-e9e6612cc4c7\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:06:04.490255 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.490159 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/61b53c91-e1dc-4f9f-942c-e9e6612cc4c7-manager-config\") pod \"lws-controller-manager-84979c6fd9-87xf5\" (UID: \"61b53c91-e1dc-4f9f-942c-e9e6612cc4c7\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:06:04.490255 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.490220 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/61b53c91-e1dc-4f9f-942c-e9e6612cc4c7-metrics-cert\") pod \"lws-controller-manager-84979c6fd9-87xf5\" (UID: \"61b53c91-e1dc-4f9f-942c-e9e6612cc4c7\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:06:04.591412 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.591368 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggpgl\" (UniqueName: \"kubernetes.io/projected/61b53c91-e1dc-4f9f-942c-e9e6612cc4c7-kube-api-access-ggpgl\") pod \"lws-controller-manager-84979c6fd9-87xf5\" (UID: \"61b53c91-e1dc-4f9f-942c-e9e6612cc4c7\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:06:04.591610 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.591430 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/61b53c91-e1dc-4f9f-942c-e9e6612cc4c7-manager-config\") pod \"lws-controller-manager-84979c6fd9-87xf5\" (UID: \"61b53c91-e1dc-4f9f-942c-e9e6612cc4c7\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:06:04.591610 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.591508 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/61b53c91-e1dc-4f9f-942c-e9e6612cc4c7-metrics-cert\") pod \"lws-controller-manager-84979c6fd9-87xf5\" (UID: \"61b53c91-e1dc-4f9f-942c-e9e6612cc4c7\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:06:04.591610 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.591555 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b53c91-e1dc-4f9f-942c-e9e6612cc4c7-cert\") pod \"lws-controller-manager-84979c6fd9-87xf5\" (UID: \"61b53c91-e1dc-4f9f-942c-e9e6612cc4c7\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:06:04.592206 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.592175 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/61b53c91-e1dc-4f9f-942c-e9e6612cc4c7-manager-config\") pod \"lws-controller-manager-84979c6fd9-87xf5\" (UID: \"61b53c91-e1dc-4f9f-942c-e9e6612cc4c7\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:06:04.594107 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.594083 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/61b53c91-e1dc-4f9f-942c-e9e6612cc4c7-metrics-cert\") pod \"lws-controller-manager-84979c6fd9-87xf5\" (UID: \"61b53c91-e1dc-4f9f-942c-e9e6612cc4c7\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:06:04.594346 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.594328 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b53c91-e1dc-4f9f-942c-e9e6612cc4c7-cert\") pod \"lws-controller-manager-84979c6fd9-87xf5\" (UID: \"61b53c91-e1dc-4f9f-942c-e9e6612cc4c7\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:06:04.622489 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.622389 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggpgl\" (UniqueName: \"kubernetes.io/projected/61b53c91-e1dc-4f9f-942c-e9e6612cc4c7-kube-api-access-ggpgl\") pod \"lws-controller-manager-84979c6fd9-87xf5\" (UID: \"61b53c91-e1dc-4f9f-942c-e9e6612cc4c7\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:06:04.667039 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.667004 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:06:04.817911 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:04.817882 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5"] Jul 06 19:06:04.819615 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:06:04.819589 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61b53c91_e1dc_4f9f_942c_e9e6612cc4c7.slice/crio-f34fa795d6b4478a4f6725a33a9dd7af65b3dae63ad50907c3f8ba7d328e2500 WatchSource:0}: Error finding container f34fa795d6b4478a4f6725a33a9dd7af65b3dae63ad50907c3f8ba7d328e2500: Status 404 returned error can't find the container with id f34fa795d6b4478a4f6725a33a9dd7af65b3dae63ad50907c3f8ba7d328e2500 Jul 06 19:06:05.241170 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:05.241135 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" event={"ID":"61b53c91-e1dc-4f9f-942c-e9e6612cc4c7","Type":"ContainerStarted","Data":"f34fa795d6b4478a4f6725a33a9dd7af65b3dae63ad50907c3f8ba7d328e2500"} Jul 06 19:06:05.428478 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:05.428432 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:06:05.784774 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:05.784725 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:06:07.236811 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.236776 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w"] Jul 06 19:06:07.239197 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.239178 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.247497 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:07.247464 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"istiod-tls\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" type="*v1.Secret" Jul 06 19:06:07.247497 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:07.247461 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"istio-kubeconfig\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" type="*v1.Secret" Jul 06 19:06:07.247651 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:07.247464 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 06 19:06:07.247651 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:07.247472 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 06 19:06:07.247651 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:07.247618 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-gw-ca-root-cert\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" type="*v1.ConfigMap" Jul 06 19:06:07.248495 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:07.248470 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cacerts\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"cacerts\"" type="*v1.Secret" Jul 06 19:06:07.248495 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:07.248477 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"istiod-openshift-gateway-94bb8fbfd-5422w\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" podUID="1fa0868b-64da-4c83-b866-29bc1a9748ee" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.248622 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:07.248482 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"istiod-openshift-gateway-dockercfg-xbd6q\" is forbidden: User \"system:node:ip-10-0-131-230.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-230.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-xbd6q\"" type="*v1.Secret" Jul 06 19:06:07.251572 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.251549 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" event={"ID":"61b53c91-e1dc-4f9f-942c-e9e6612cc4c7","Type":"ContainerStarted","Data":"8f679c1b05720d38c60f3f9dd2fd531d7f2a6291b08baacea0839fa6a30595b2"} Jul 06 19:06:07.251628 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.251608 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:06:07.275349 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.275314 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w"] Jul 06 19:06:07.302108 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.302051 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" podStartSLOduration=1.610576572 podStartE2EDuration="3.302036094s" totalImagesPullingTime="1.691459522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:06:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:06:07.301361819 +0000 UTC m=+407.662525612" watchObservedRunningTime="2026-07-06 19:06:07.302036094 +0000 UTC m=+407.663199887" Jul 06 19:06:07.318812 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.318768 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.318812 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.318812 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.319073 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.318878 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jwxh\" (UniqueName: \"kubernetes.io/projected/1fa0868b-64da-4c83-b866-29bc1a9748ee-kube-api-access-4jwxh\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.319073 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.318909 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.319073 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.318950 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.319073 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.319036 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa0868b-64da-4c83-b866-29bc1a9748ee-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.319073 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.319063 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.420056 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.420015 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jwxh\" (UniqueName: \"kubernetes.io/projected/1fa0868b-64da-4c83-b866-29bc1a9748ee-kube-api-access-4jwxh\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.420308 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.420070 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.420308 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.420094 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.420308 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.420212 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa0868b-64da-4c83-b866-29bc1a9748ee-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.420308 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.420246 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.420551 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.420313 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.420551 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.420339 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.422521 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.422495 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa0868b-64da-4c83-b866-29bc1a9748ee-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.461723 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.461691 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:07.466249 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.466219 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:06:07.772961 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.772923 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:06:07.989508 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:07.989474 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:06:08.191383 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:08.191291 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-xbd6q\"" Jul 06 19:06:08.358247 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:08.358213 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 19:06:08.421080 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:08.421041 2557 configmap.go:193] Couldn't get configMap openshift-ingress/openshift-gw-ca-root-cert: failed to sync configmap cache: timed out waiting for the condition Jul 06 19:06:08.421080 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:08.421060 2557 secret.go:189] Couldn't get secret openshift-ingress/cacerts: failed to sync secret cache: timed out waiting for the condition Jul 06 19:06:08.421292 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:08.421086 2557 secret.go:189] Couldn't get secret openshift-ingress/istio-kubeconfig: failed to sync secret cache: timed out waiting for the condition Jul 06 19:06:08.421292 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:08.421125 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-cacerts podName:1fa0868b-64da-4c83-b866-29bc1a9748ee nodeName:}" failed. No retries permitted until 2026-07-06 19:06:08.921108015 +0000 UTC m=+409.282271787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cacerts" (UniqueName: "kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-cacerts") pod "istiod-openshift-gateway-94bb8fbfd-5422w" (UID: "1fa0868b-64da-4c83-b866-29bc1a9748ee") : failed to sync secret cache: timed out waiting for the condition Jul 06 19:06:08.421292 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:08.421145 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-csr-ca-configmap podName:1fa0868b-64da-4c83-b866-29bc1a9748ee nodeName:}" failed. No retries permitted until 2026-07-06 19:06:08.921135835 +0000 UTC m=+409.282299604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "istio-csr-ca-configmap" (UniqueName: "kubernetes.io/configmap/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-csr-ca-configmap") pod "istiod-openshift-gateway-94bb8fbfd-5422w" (UID: "1fa0868b-64da-4c83-b866-29bc1a9748ee") : failed to sync configmap cache: timed out waiting for the condition Jul 06 19:06:08.421292 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:08.421050 2557 secret.go:189] Couldn't get secret openshift-ingress/istiod-tls: failed to sync secret cache: timed out waiting for the condition Jul 06 19:06:08.421292 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:08.421157 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-kubeconfig podName:1fa0868b-64da-4c83-b866-29bc1a9748ee nodeName:}" failed. No retries permitted until 2026-07-06 19:06:08.921151648 +0000 UTC m=+409.282315418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "istio-kubeconfig" (UniqueName: "kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-kubeconfig") pod "istiod-openshift-gateway-94bb8fbfd-5422w" (UID: "1fa0868b-64da-4c83-b866-29bc1a9748ee") : failed to sync secret cache: timed out waiting for the condition Jul 06 19:06:08.421292 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:08.421213 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-csr-dns-cert podName:1fa0868b-64da-4c83-b866-29bc1a9748ee nodeName:}" failed. No retries permitted until 2026-07-06 19:06:08.921195118 +0000 UTC m=+409.282358902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "istio-csr-dns-cert" (UniqueName: "kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-csr-dns-cert") pod "istiod-openshift-gateway-94bb8fbfd-5422w" (UID: "1fa0868b-64da-4c83-b866-29bc1a9748ee") : failed to sync secret cache: timed out waiting for the condition Jul 06 19:06:08.441807 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:08.441732 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 19:06:08.443229 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:08.443201 2557 projected.go:291] Couldn't get configMap openshift-ingress/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 06 19:06:08.443349 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:08.443250 2557 projected.go:196] Error preparing data for projected volume kube-api-access-4jwxh for pod openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w: failed to sync configmap cache: timed out waiting for the condition Jul 06 19:06:08.443349 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:08.443319 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fa0868b-64da-4c83-b866-29bc1a9748ee-kube-api-access-4jwxh podName:1fa0868b-64da-4c83-b866-29bc1a9748ee nodeName:}" failed. No retries permitted until 2026-07-06 19:06:08.943298576 +0000 UTC m=+409.304462346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4jwxh" (UniqueName: "kubernetes.io/projected/1fa0868b-64da-4c83-b866-29bc1a9748ee-kube-api-access-4jwxh") pod "istiod-openshift-gateway-94bb8fbfd-5422w" (UID: "1fa0868b-64da-4c83-b866-29bc1a9748ee") : failed to sync configmap cache: timed out waiting for the condition Jul 06 19:06:08.751122 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:08.751095 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 19:06:08.759689 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:08.759661 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 19:06:08.771031 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:08.771008 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 19:06:08.822804 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:08.822781 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 19:06:08.934957 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:08.934920 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:08.935165 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:08.934975 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:08.935165 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:08.934997 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:08.935165 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:08.935055 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:08.935687 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:08.935662 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:08.937464 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:08.937419 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:08.937592 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:08.937424 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:08.937592 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:08.937571 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1fa0868b-64da-4c83-b866-29bc1a9748ee-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:09.035729 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:09.035648 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jwxh\" (UniqueName: \"kubernetes.io/projected/1fa0868b-64da-4c83-b866-29bc1a9748ee-kube-api-access-4jwxh\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:09.038139 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:09.038116 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jwxh\" (UniqueName: \"kubernetes.io/projected/1fa0868b-64da-4c83-b866-29bc1a9748ee-kube-api-access-4jwxh\") pod \"istiod-openshift-gateway-94bb8fbfd-5422w\" (UID: \"1fa0868b-64da-4c83-b866-29bc1a9748ee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:09.048935 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:09.048912 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:09.202416 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:09.202381 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w"] Jul 06 19:06:09.226997 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:06:09.226968 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fa0868b_64da_4c83_b866_29bc1a9748ee.slice/crio-4474d050ea62ac1f0cd83695d27abbd2450651e42da6a39297c71f8334bf1e8d WatchSource:0}: Error finding container 4474d050ea62ac1f0cd83695d27abbd2450651e42da6a39297c71f8334bf1e8d: Status 404 returned error can't find the container with id 4474d050ea62ac1f0cd83695d27abbd2450651e42da6a39297c71f8334bf1e8d Jul 06 19:06:09.262280 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:09.262242 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" event={"ID":"1fa0868b-64da-4c83-b866-29bc1a9748ee","Type":"ContainerStarted","Data":"4474d050ea62ac1f0cd83695d27abbd2450651e42da6a39297c71f8334bf1e8d"} Jul 06 19:06:09.734915 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:09.734878 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b4c9dc875-lf457"] Jul 06 19:06:10.138666 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:10.138630 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:06:10.340581 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:10.340546 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:06:12.471237 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:12.471200 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 19:06:12.471481 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:12.471271 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 19:06:13.285323 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:13.285288 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" event={"ID":"1fa0868b-64da-4c83-b866-29bc1a9748ee","Type":"ContainerStarted","Data":"4ed32f7413f908cd353f4607431097bc021f062c869a9eb04a7b99ed6cf87962"} Jul 06 19:06:13.285570 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:13.285486 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:13.287118 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:13.287093 2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-5422w container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 19:06:13.287217 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:13.287135 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" podUID="1fa0868b-64da-4c83-b866-29bc1a9748ee" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 19:06:14.028640 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:14.028579 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" podStartSLOduration=3.7867071340000003 podStartE2EDuration="7.02856261s" totalImagesPullingTime="3.241855476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:06:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:06:13.998185956 +0000 UTC m=+414.359349750" watchObservedRunningTime="2026-07-06 19:06:14.02856261 +0000 UTC m=+414.389726403" Jul 06 19:06:14.291136 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:14.291040 2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-5422w container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 19:06:14.291136 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:14.291105 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" podUID="1fa0868b-64da-4c83-b866-29bc1a9748ee" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 19:06:17.292020 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:17.291991 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5422w" Jul 06 19:06:18.259089 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:18.259053 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:06:18.805976 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:18.805936 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:06:18.963844 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:18.963750 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:06:34.754610 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:34.754542 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7b4c9dc875-lf457" podUID="6dec7f68-ed04-49be-a571-68a52ecfa052" containerName="console" containerID="cri-o://50585cb194b15b330507a7f0e303f5a1bb6d2fe1c8712dc79d251b6f5d5a5c5e" gracePeriod=15 Jul 06 19:06:35.000313 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.000289 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:06:35.075776 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.075691 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dec7f68-ed04-49be-a571-68a52ecfa052-console-serving-cert\") pod \"6dec7f68-ed04-49be-a571-68a52ecfa052\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " Jul 06 19:06:35.075776 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.075730 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-oauth-serving-cert\") pod \"6dec7f68-ed04-49be-a571-68a52ecfa052\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " Jul 06 19:06:35.075990 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.075781 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dec7f68-ed04-49be-a571-68a52ecfa052-console-oauth-config\") pod \"6dec7f68-ed04-49be-a571-68a52ecfa052\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " Jul 06 19:06:35.075990 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.075802 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-service-ca\") pod \"6dec7f68-ed04-49be-a571-68a52ecfa052\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " Jul 06 19:06:35.075990 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.075826 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mkbf\" (UniqueName: \"kubernetes.io/projected/6dec7f68-ed04-49be-a571-68a52ecfa052-kube-api-access-9mkbf\") pod \"6dec7f68-ed04-49be-a571-68a52ecfa052\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " Jul 06 19:06:35.075990 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.075902 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-console-config\") pod \"6dec7f68-ed04-49be-a571-68a52ecfa052\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " Jul 06 19:06:35.075990 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.075987 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-trusted-ca-bundle\") pod \"6dec7f68-ed04-49be-a571-68a52ecfa052\" (UID: \"6dec7f68-ed04-49be-a571-68a52ecfa052\") " Jul 06 19:06:35.076290 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.076266 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6dec7f68-ed04-49be-a571-68a52ecfa052" (UID: "6dec7f68-ed04-49be-a571-68a52ecfa052"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:06:35.076290 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.076269 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-service-ca" (OuterVolumeSpecName: "service-ca") pod "6dec7f68-ed04-49be-a571-68a52ecfa052" (UID: "6dec7f68-ed04-49be-a571-68a52ecfa052"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:06:35.076392 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.076294 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-console-config" (OuterVolumeSpecName: "console-config") pod "6dec7f68-ed04-49be-a571-68a52ecfa052" (UID: "6dec7f68-ed04-49be-a571-68a52ecfa052"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:06:35.076740 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.076714 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6dec7f68-ed04-49be-a571-68a52ecfa052" (UID: "6dec7f68-ed04-49be-a571-68a52ecfa052"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:06:35.078107 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.078086 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dec7f68-ed04-49be-a571-68a52ecfa052-kube-api-access-9mkbf" (OuterVolumeSpecName: "kube-api-access-9mkbf") pod "6dec7f68-ed04-49be-a571-68a52ecfa052" (UID: "6dec7f68-ed04-49be-a571-68a52ecfa052"). InnerVolumeSpecName "kube-api-access-9mkbf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:06:35.078367 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.078344 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dec7f68-ed04-49be-a571-68a52ecfa052-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6dec7f68-ed04-49be-a571-68a52ecfa052" (UID: "6dec7f68-ed04-49be-a571-68a52ecfa052"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:06:35.078417 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.078372 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dec7f68-ed04-49be-a571-68a52ecfa052-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6dec7f68-ed04-49be-a571-68a52ecfa052" (UID: "6dec7f68-ed04-49be-a571-68a52ecfa052"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:06:35.177806 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.177752 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dec7f68-ed04-49be-a571-68a52ecfa052-console-serving-cert\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:06:35.177806 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.177800 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-oauth-serving-cert\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:06:35.177806 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.177810 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dec7f68-ed04-49be-a571-68a52ecfa052-console-oauth-config\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:06:35.177806 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.177822 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-service-ca\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:06:35.178085 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.177832 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mkbf\" (UniqueName: \"kubernetes.io/projected/6dec7f68-ed04-49be-a571-68a52ecfa052-kube-api-access-9mkbf\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:06:35.178085 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.177841 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-console-config\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:06:35.178085 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.177849 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dec7f68-ed04-49be-a571-68a52ecfa052-trusted-ca-bundle\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:06:35.387637 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.387546 2557 generic.go:356] "Generic (PLEG): container finished" podID="6dec7f68-ed04-49be-a571-68a52ecfa052" containerID="50585cb194b15b330507a7f0e303f5a1bb6d2fe1c8712dc79d251b6f5d5a5c5e" exitCode=0 Jul 06 19:06:35.387637 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.387592 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b4c9dc875-lf457" Jul 06 19:06:35.387637 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.387628 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b4c9dc875-lf457" event={"ID":"6dec7f68-ed04-49be-a571-68a52ecfa052","Type":"ContainerDied","Data":"50585cb194b15b330507a7f0e303f5a1bb6d2fe1c8712dc79d251b6f5d5a5c5e"} Jul 06 19:06:35.387912 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.387683 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b4c9dc875-lf457" event={"ID":"6dec7f68-ed04-49be-a571-68a52ecfa052","Type":"ContainerDied","Data":"479a5e29494192ae8a9654a289f780cb2f5f7fe7016db41f2ad7c2e8c61fe27d"} Jul 06 19:06:35.387912 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.387708 2557 scope.go:117] "RemoveContainer" containerID="50585cb194b15b330507a7f0e303f5a1bb6d2fe1c8712dc79d251b6f5d5a5c5e" Jul 06 19:06:35.396873 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.396851 2557 scope.go:117] "RemoveContainer" containerID="50585cb194b15b330507a7f0e303f5a1bb6d2fe1c8712dc79d251b6f5d5a5c5e" Jul 06 19:06:35.397170 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:06:35.397150 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50585cb194b15b330507a7f0e303f5a1bb6d2fe1c8712dc79d251b6f5d5a5c5e\": container with ID starting with 50585cb194b15b330507a7f0e303f5a1bb6d2fe1c8712dc79d251b6f5d5a5c5e not found: ID does not exist" containerID="50585cb194b15b330507a7f0e303f5a1bb6d2fe1c8712dc79d251b6f5d5a5c5e" Jul 06 19:06:35.397222 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.397179 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50585cb194b15b330507a7f0e303f5a1bb6d2fe1c8712dc79d251b6f5d5a5c5e"} err="failed to get container status \"50585cb194b15b330507a7f0e303f5a1bb6d2fe1c8712dc79d251b6f5d5a5c5e\": rpc error: code = NotFound desc = could not find container \"50585cb194b15b330507a7f0e303f5a1bb6d2fe1c8712dc79d251b6f5d5a5c5e\": container with ID starting with 50585cb194b15b330507a7f0e303f5a1bb6d2fe1c8712dc79d251b6f5d5a5c5e not found: ID does not exist" Jul 06 19:06:35.411124 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.411094 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b4c9dc875-lf457"] Jul 06 19:06:35.418045 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:35.418015 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b4c9dc875-lf457"] Jul 06 19:06:36.197023 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:06:36.196985 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dec7f68-ed04-49be-a571-68a52ecfa052" path="/var/lib/kubelet/pods/6dec7f68-ed04-49be-a571-68a52ecfa052/volumes" Jul 06 19:07:17.615929 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:07:17.615889 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:17.855307 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:07:17.855269 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:18.076000 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:07:18.075962 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:18.485735 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:07:18.485644 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:18.672396 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:07:18.672350 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:47.395450 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:07:47.395392 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:50.878297 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:07:50.878256 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:54.008906 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:07:54.008869 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:55.414972 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:07:55.414922 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:56.326324 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:07:56.326287 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:57.287918 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:07:57.287884 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:58.637937 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:07:58.637896 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:58.920156 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:07:58.920054 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:59.111388 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:07:59.111347 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:59.286732 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:07:59.286698 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:02.499512 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:02.499463 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:02.647688 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:02.647614 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:06.288148 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:06.288112 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:06.445660 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:06.445605 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:06.598013 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:06.597919 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:14.165132 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:14.165045 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:14.457263 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:14.457183 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:14.708301 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:14.708207 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:15.044988 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:15.044955 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:17.460822 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:17.460790 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr"] Jul 06 19:08:17.461343 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:17.461327 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dec7f68-ed04-49be-a571-68a52ecfa052" containerName="console" Jul 06 19:08:17.461628 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:17.461347 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dec7f68-ed04-49be-a571-68a52ecfa052" containerName="console" Jul 06 19:08:17.461628 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:17.461471 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dec7f68-ed04-49be-a571-68a52ecfa052" containerName="console" Jul 06 19:08:17.464314 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:17.464293 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:08:17.476284 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:17.476257 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-mvgvw\"" Jul 06 19:08:17.476284 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:17.476277 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 19:08:17.476515 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:17.476494 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 19:08:17.500657 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:17.500626 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr"] Jul 06 19:08:17.608535 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:17.608503 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccdcr\" (UniqueName: \"kubernetes.io/projected/b2d540f2-9dec-4d75-842d-815689156ee0-kube-api-access-ccdcr\") pod \"limitador-operator-controller-manager-859d7b78bd-gd8lr\" (UID: \"b2d540f2-9dec-4d75-842d-815689156ee0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:08:17.709026 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:17.708982 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccdcr\" (UniqueName: \"kubernetes.io/projected/b2d540f2-9dec-4d75-842d-815689156ee0-kube-api-access-ccdcr\") pod \"limitador-operator-controller-manager-859d7b78bd-gd8lr\" (UID: \"b2d540f2-9dec-4d75-842d-815689156ee0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:08:17.741735 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:17.741704 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccdcr\" (UniqueName: \"kubernetes.io/projected/b2d540f2-9dec-4d75-842d-815689156ee0-kube-api-access-ccdcr\") pod \"limitador-operator-controller-manager-859d7b78bd-gd8lr\" (UID: \"b2d540f2-9dec-4d75-842d-815689156ee0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:08:17.774579 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:17.774539 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:08:18.028010 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:18.027926 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr"] Jul 06 19:08:18.031238 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:08:18.031207 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d540f2_9dec_4d75_842d_815689156ee0.slice/crio-7f54e079ba4eb5e44189eed364265acaa15992bdb387bc0265c13aa1f0526861 WatchSource:0}: Error finding container 7f54e079ba4eb5e44189eed364265acaa15992bdb387bc0265c13aa1f0526861: Status 404 returned error can't find the container with id 7f54e079ba4eb5e44189eed364265acaa15992bdb387bc0265c13aa1f0526861 Jul 06 19:08:18.872229 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:18.872195 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" event={"ID":"b2d540f2-9dec-4d75-842d-815689156ee0","Type":"ContainerStarted","Data":"7f54e079ba4eb5e44189eed364265acaa15992bdb387bc0265c13aa1f0526861"} Jul 06 19:08:19.508446 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:19.508408 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:19.820874 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:19.820778 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:20.175747 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:20.175684 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:20.883632 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:20.883592 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" event={"ID":"b2d540f2-9dec-4d75-842d-815689156ee0","Type":"ContainerStarted","Data":"cf28b03bb6d11ca6b332bb06db10f418d675e1181d31ea791e7851c9be0b65bd"} Jul 06 19:08:20.883857 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:20.883732 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:08:20.972724 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:20.972666 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" podStartSLOduration=1.8797781709999999 podStartE2EDuration="3.972650255s" totalImagesPullingTime="2.092872084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:08:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:08:20.971075125 +0000 UTC m=+541.332238914" watchObservedRunningTime="2026-07-06 19:08:20.972650255 +0000 UTC m=+541.333814046" Jul 06 19:08:22.289905 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:22.289858 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:22.509593 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:22.509552 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:27.685743 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:27.685700 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:28.118536 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:28.118502 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:31.891139 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:31.891107 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:08:36.435750 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:36.435713 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:36.985162 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:36.985111 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:40.833177 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:40.833134 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:41.039690 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:41.039657 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:41.204199 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:41.204092 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:45.576640 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:45.576597 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:46.752288 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:46.752253 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:52.143133 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:08:52.143097 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:11.890153 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:11.890107 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" podUID="b2d540f2-9dec-4d75-842d-815689156ee0" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.30:8081/readyz\": dial tcp 10.134.0.30:8081: connect: connection refused" Jul 06 19:09:12.130165 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:12.130128 2557 generic.go:356] "Generic (PLEG): container finished" podID="b2d540f2-9dec-4d75-842d-815689156ee0" containerID="cf28b03bb6d11ca6b332bb06db10f418d675e1181d31ea791e7851c9be0b65bd" exitCode=1 Jul 06 19:09:12.130339 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:12.130187 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" event={"ID":"b2d540f2-9dec-4d75-842d-815689156ee0","Type":"ContainerDied","Data":"cf28b03bb6d11ca6b332bb06db10f418d675e1181d31ea791e7851c9be0b65bd"} Jul 06 19:09:12.130540 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:12.130527 2557 scope.go:117] "RemoveContainer" containerID="cf28b03bb6d11ca6b332bb06db10f418d675e1181d31ea791e7851c9be0b65bd" Jul 06 19:09:12.665539 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:09:12.665493 2557 controller.go:195] "Failed to update lease" err="Put \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-131-230.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 06 19:09:13.137064 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:13.137027 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" event={"ID":"b2d540f2-9dec-4d75-842d-815689156ee0","Type":"ContainerStarted","Data":"0f3252e4a72a62b3859791f134fc8d2006f30019b0d3512331a89f885b4fb9a7"} Jul 06 19:09:13.137469 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:13.137231 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:09:14.055376 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:09:14.055343 2557 controller.go:195] "Failed to update lease" err="Operation cannot be fulfilled on leases.coordination.k8s.io \"ip-10-0-131-230.ec2.internal\": the object has been modified; please apply your changes to the latest version and try again" Jul 06 19:09:14.169571 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:14.169537 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:15.131921 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:15.131881 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:15.331199 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:15.331162 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:15.461908 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:15.461806 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:24.148649 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:24.148620 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:09:34.177359 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:09:34.177307 2557 controller.go:195] "Failed to update lease" err="Put \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-131-230.ec2.internal?timeout=10s\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 19:09:35.412275 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:09:35.412240 2557 controller.go:195] "Failed to update lease" err="Operation cannot be fulfilled on leases.coordination.k8s.io \"ip-10-0-131-230.ec2.internal\": the object has been modified; please apply your changes to the latest version and try again" Jul 06 19:09:35.575986 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:35.575950 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:35.776751 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:35.776722 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:35.923323 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:35.923291 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:37.275705 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:37.275670 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:37.563680 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:37.563584 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:38.328328 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:38.328289 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:38.439385 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:38.439346 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:39.298373 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:39.298336 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:39.536153 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:39.536112 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:53.244194 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:53.244144 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:59.507671 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:09:59.507628 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:09.157759 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:09.157720 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:09.251637 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:09.251600 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:12.535261 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:12.535229 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-nzrj5"] Jul 06 19:10:12.538798 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:12.538782 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-nzrj5" Jul 06 19:10:12.543334 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:12.543315 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 06 19:10:12.543469 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:12.543349 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-jlb4q\"" Jul 06 19:10:12.565348 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:12.565322 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-nzrj5"] Jul 06 19:10:12.675799 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:12.675764 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:12.681866 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:12.681838 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0c7f699e-49b9-4132-8c15-b0046803b3ad-data\") pod \"postgres-69fb4594cf-nzrj5\" (UID: \"0c7f699e-49b9-4132-8c15-b0046803b3ad\") " pod="opendatahub/postgres-69fb4594cf-nzrj5" Jul 06 19:10:12.682006 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:12.681908 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpr8l\" (UniqueName: \"kubernetes.io/projected/0c7f699e-49b9-4132-8c15-b0046803b3ad-kube-api-access-bpr8l\") pod \"postgres-69fb4594cf-nzrj5\" (UID: \"0c7f699e-49b9-4132-8c15-b0046803b3ad\") " pod="opendatahub/postgres-69fb4594cf-nzrj5" Jul 06 19:10:12.783315 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:12.783276 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpr8l\" (UniqueName: \"kubernetes.io/projected/0c7f699e-49b9-4132-8c15-b0046803b3ad-kube-api-access-bpr8l\") pod \"postgres-69fb4594cf-nzrj5\" (UID: \"0c7f699e-49b9-4132-8c15-b0046803b3ad\") " pod="opendatahub/postgres-69fb4594cf-nzrj5" Jul 06 19:10:12.783548 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:12.783373 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0c7f699e-49b9-4132-8c15-b0046803b3ad-data\") pod \"postgres-69fb4594cf-nzrj5\" (UID: \"0c7f699e-49b9-4132-8c15-b0046803b3ad\") " pod="opendatahub/postgres-69fb4594cf-nzrj5" Jul 06 19:10:12.783844 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:12.783822 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0c7f699e-49b9-4132-8c15-b0046803b3ad-data\") pod \"postgres-69fb4594cf-nzrj5\" (UID: \"0c7f699e-49b9-4132-8c15-b0046803b3ad\") " pod="opendatahub/postgres-69fb4594cf-nzrj5" Jul 06 19:10:12.806712 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:12.806634 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpr8l\" (UniqueName: \"kubernetes.io/projected/0c7f699e-49b9-4132-8c15-b0046803b3ad-kube-api-access-bpr8l\") pod \"postgres-69fb4594cf-nzrj5\" (UID: \"0c7f699e-49b9-4132-8c15-b0046803b3ad\") " pod="opendatahub/postgres-69fb4594cf-nzrj5" Jul 06 19:10:12.849370 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:12.849335 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-nzrj5" Jul 06 19:10:12.994769 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:12.994745 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-nzrj5"] Jul 06 19:10:12.997765 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:10:12.997735 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c7f699e_49b9_4132_8c15_b0046803b3ad.slice/crio-252ac8b8b3120d61c1b42ec1dfea398eea584cbfe816ea0e4073c6018a8f45fb WatchSource:0}: Error finding container 252ac8b8b3120d61c1b42ec1dfea398eea584cbfe816ea0e4073c6018a8f45fb: Status 404 returned error can't find the container with id 252ac8b8b3120d61c1b42ec1dfea398eea584cbfe816ea0e4073c6018a8f45fb Jul 06 19:10:12.999190 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:12.999175 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:10:13.429623 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:13.429588 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-nzrj5" event={"ID":"0c7f699e-49b9-4132-8c15-b0046803b3ad","Type":"ContainerStarted","Data":"252ac8b8b3120d61c1b42ec1dfea398eea584cbfe816ea0e4073c6018a8f45fb"} Jul 06 19:10:14.077368 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:14.077317 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:14.204673 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:14.204515 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:18.460961 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:18.460928 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-nzrj5" event={"ID":"0c7f699e-49b9-4132-8c15-b0046803b3ad","Type":"ContainerStarted","Data":"2cc4804ea0aaea15eae420a3c1053ead53eb9128614cefdc896c6e910012a1b5"} Jul 06 19:10:18.461414 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:18.461009 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-nzrj5" Jul 06 19:10:18.500378 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:18.500322 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-nzrj5" podStartSLOduration=1.69476212 podStartE2EDuration="6.500300777s" totalImagesPullingTime="4.805538657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:18.498679951 +0000 UTC m=+658.859843742" watchObservedRunningTime="2026-07-06 19:10:18.500300777 +0000 UTC m=+658.861464571" Jul 06 19:10:24.496788 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:24.496751 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-nzrj5" Jul 06 19:10:32.127499 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:10:32.127268 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:10:22Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:10:22Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:10:22Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:10:22Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a70169bf82777e89a725ee06c18161a796d50f5dc09d3b1980d8cf15b5c48b7b\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b90fb00a6f277d9e99d519aaa9cd2efb6f3d1c8b48f444d5135c733346c76b90\\\"],\\\"sizeBytes\\\":2092264135},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:511f5069a16301bd6df533513412d0ca308fcad1edf5859bc53cbc973f17d6df\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\"],\\\"sizeBytes\\\":1438655886},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:55ec7147f859301bbb8288966c8a21c5db729200f3c3d91db671978baef79517\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d87af67c8f369963bc01220c5cf1d56af737b9bcd46947398a3128b7137e1d1a\\\"],\\\"sizeBytes\\\":1208832350},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1537aadb2d8808accf423891ceb3a6a2deb0111bc02666f0c8df5591d7abc92\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f5c00c67012fedc41b1c472dbaa9d6dece63c9a82747da62b3243fa91cad3c35\\\"],\\\"sizeBytes\\\":901961931},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5be8cdcf34f35a89a70b6bfa49e66897641871237570b3e09766f83a0a473e6a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5dd08f95712ac6420fe74f5fe5689a11a8b4114a22e0d84dc2d76eed6bd4f4fe\\\"],\\\"sizeBytes\\\":612408602},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1cbdec18368e5cf46bfda529c7e8a8ad32261b64d0b421ec69da4a3b9868751d\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2d26421a926916d11490ce5dbff564dad6e8978a5e5fa392a63b25beca2848aa\\\"],\\\"sizeBytes\\\":606240501},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2d5a37efb4a917780398d375b7156193497f4554ae216ef110a445724d665111\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:af46126bb56c6e82bf5610e51889b99320ba681c23ea776c7533c233f2a3c5b7\\\"],\\\"sizeBytes\\\":601507442},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c578e9a2a8e1e9b1c1adfea042f3a559baf1dcc0a601d47f2ab2360af87e7a74\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1d4a9e325ceca01359ab5b6ec7f7dcac51f1c76b1721d214aec0e01820ab592\\\"],\\\"sizeBytes\\\":563573142},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3901dd6834e7bed4942e1e4ceb7f4817615d534ccdd274aba200bbc587183585\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0f6241919ad7cdf81e4b35775946f82ea8fa362d39096a34b3ae3de7c003904\\\"],\\\"sizeBytes\\\":500548708},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0dc1d2bf1c7b77613eb12dc3dec7323cfde9a04673ea7ed31edf4da775c22642\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a9b1c2b280c23e90adf1d0d669ca39e0d6829205fbe27e725afe5c35e0037ca6\\\"],\\\"sizeBytes\\\":482011914},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6082ff851706c560884555c640798ce7a028ddf6b3a3cfa2f7579323f858a826\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:da9a648c75766e888f7d5037d75344f75b1d319d011bc5217e0d3ba3d5071a88\\\"],\\\"sizeBytes\\\":463915795},{\\\"names\\\":[\\\"quay.io/openshift/origin-insights-runtime-extractor@sha256:c4f7b0c6eefe8b18e6d8a4bc11b3b70f951f811dd551b392ebf235730a8c1e97\\\",\\\"quay.io/openshift/origin-insights-runtime-extractor@sha256:d6a4414d94942241301f29743a489515ee3bf951d9277de040d20882cd45204f\\\",\\\"quay.io/openshift/origin-insights-runtime-extractor:latest\\\"],\\\"sizeBytes\\\":421414665},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4e326414ebdae7f05b02e7778707689b9cd9638fa71e5737357fc43b790c9bdd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d3e33a8c1a0a582f1e62f55562b0da406d0675b6f677e046cff78401afcb7544\\\"],\\\"sizeBytes\\\":412532612},{\\\"names\\\":[\\\"registry.redhat.io/rhel9/postgresql-16@sha256:1149c78dbc442a419d6d9073dd5b60dd40d5639070b8cd8cd8dfab30ef88a913\\\",\\\"registry.redhat.io/rhel9/postgresql-16@sha256:cb66eb4b438ce3f9db194dd1948ce25dc4142e789025f214324b7baa3c915c05\\\",\\\"registry.redhat.io/rhel9/postgresql-16:latest\\\"],\\\"sizeBytes\\\":403303517},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:51c853fa1a08088e44a396ed8321c824b483bab3673f8a060ba5732a88deb565\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8cde2dd29b991fab261b34481407faace228f79f1e92bdeef6c97eec1af7538e\\\"],\\\"sizeBytes\\\":375179023},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:28389dd118629aa9d7c2525202f3ad6359afb98b6a3bf7672cb1ecfae2418e23\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:93d5659f2d5779d0df45117065725d0184040149328a4ce4b5e59bb2d06e230a\\\"],\\\"sizeBytes\\\":372388955},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1152d0088e99aeef75203e7e9185507833379e9829161a9f78af5500ca944f46\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:49287a1683e255a8c7d468520a3bd9e589ec8c38863ecc28ce4886ed0cce9828\\\"],\\\"sizeBytes\\\":368851365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:81fb18e5930816a580cbd2c3efe355f747b5d7341242637e44fa7182ee079800\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c1b749066725367c3e651210210d90a9e389d1f4ba4c20431ab4cab5001b31a0\\\"],\\\"sizeBytes\\\":350735931},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:022f1a802b21c79f11da33266bd78688e33f8fa5cd49189e8314ad987e1046bc\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:437a5fd01dd4bfbdd635d5069df3ee7ee9095636618f521f8c8f35dc792e75ca\\\"],\\\"sizeBytes\\\":343923425},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d6df0cb6282bc5ba571a8de322fe598afdf22e817f5a7dc92db3274dbdb9a1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:534df4b689087b5ab447fe3e97601dfb9617ea784c05dfbbde5e6a3ebc15e929\\\"],\\\"sizeBytes\\\":342879910},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1c2a3f6980952094d9116ab4ae3fbff42dc54c9a4ee9a031c31ef6b4d9d5cb9b\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:60d76d51dac424f46a693b8b319b38803b48ebb091938c4153ffa0fafb50fd11\\\"],\\\"sizeBytes\\\":338739808},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6881867b07165b6f1d0161f646d6d4ebfe2d9979327b482e6f8df0fd4c412cbe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d95a1c6f868ae4337487acd1b7239a8f40c2905b7574570d66655a2caf268ad6\\\"],\\\"sizeBytes\\\":325729914},{\\\"names\\\":[\\\"quay.io/openshift/origin-kube-rbac-proxy@sha256:9fa7c0125c5217797dcd586e7cc44e9c1d6a8f931bcb4d3eddaa2d1b0c27e2ce\\\",\\\"quay.io/openshift/origin-kube-rbac-proxy@sha256:d82c862da8fe5446edc0eb8fbed1f1f6bc38aac7d7c1586cad0e03d3efd6f90e\\\",\\\"quay.io/openshift/origin-kube-rbac-proxy:latest\\\"],\\\"sizeBytes\\\":325671898},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e3d4671bf7f2003bfb95459b9e5e3167d51966c587c887b63c067e9a6001d09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eb2912aa612e75e08046f10aa8408f79e2578917c8ce1f17ec0772a2bd4f02a2\\\"],\\\"sizeBytes\\\":314111041},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:00aaa47e1b08f63b4600d9c19fd0b7f3d100d4b022fd98493aebf85d831b3693\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b2e954d9267f62174ff45a44976fa7a47e58d87fef14ffbb9fc0f8184e722280\\\"],\\\"sizeBytes\\\":309676442},{\\\"names\\\":[\\\"registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:681a5f3e02131df294a2b83933584403fe4035d43d98530a13f83a77f71171fa\\\",\\\"registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:f3683d29aa703829fb0d0ba93711188fb717cc1ac156877c9d0ab5fb5083513f\\\"],\\\"sizeBytes\\\":308936624},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:599e9f399a5256df7848dc86f5a7c14748dbeff49d3644fd7935489d07072e0e\\\"],\\\"sizeBytes\\\":303912755},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:919cd188941de1705b0b309755fd19c065999938b4c561ad08e8476ea9116f44\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6023b6bce522f459b4e662ae637701870f6296d329e13d27e05f602cbb56096\\\"],\\\"sizeBytes\\\":300938814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:038e50a65e9e4447cb568b8672318f02bbd9376d10c835bb0e01a3126721d8bd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fa6250524898eaddbd8f18dcc345df6d71238b0ebc4e0598485057ef7aee2f67\\\"],\\\"sizeBytes\\\":295995367},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:103b203392b4f83ec0554d1bc91ddea428aa328719707803ce8551a1d1aabd32\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eb5c1bc090355919da7a3f8a1f4f3f66244e61a0d7ac927c545faa373ead6905\\\"],\\\"sizeBytes\\\":293810026},{\\\"names\\\":[\\\"quay.io/opendatahub/opendatahub-operator@sha256:62da7505a0bd1ed943c21ede46ffb0281020eba3b56657307e73ccf96a4ee006\\\",\\\"quay.io/opendatahub/opendatahub-operator@sha256:69b729c6ba9c3d7f1f9c604d5d6f10079c9997224552da5dca3186e90f7512fd\\\",\\\"quay.io/opendatahub/opendatahub-operator:v3.5.0-ea.2\\\"],\\\"sizeBytes\\\":293029345},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:25d3a4a674f67b96b494f454f4a0bbc2b242afd1990d734e821a905ea1b6b429\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5079f558a4811d9761daaf08dd2bff84fbfe9ac5ae6a91ed8ea7c8f899605427\\\"],\\\"sizeBytes\\\":292347304},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:773bab9410669d654b42e5d3cceca5989c5be7652ef9f6ff72179ae0c63e0dac\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d6927f980baf4af326e1a723be59b315894ba986ea4fb5e9fe9cb045575dc2b7\\\"],\\\"sizeBytes\\\":276586373},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:121de0f318d50621c54c9ceedd72d975745cdef8adbfce041113cd3ef3543090\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:54daab5a81cfe8c31382d27a41095dcd8db35bc07acdc90f3cede90ffc06a782\\\"],\\\"sizeBytes\\\":275908478},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e7040afaaa68ee77d97ff176987a40ae096a2808c0b732796fd9512e980384d7\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fff38eefce4202e98d3f93d274a73c576403ab43d85609f72f414922ada6cf50\\\"],\\\"sizeBytes\\\":274511508},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:61ec103ea7e6351f76851bc82db635db95646d2528da39cf6093bce74d33338c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6aa8e24edbb4f884870dd2b5189e19d18884586cb62f391f1dd32891283482d4\\\"],\\\"sizeBytes\\\":273315827},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d77ed4b5716efb9fea719ee357ea26d99436291ac7e648c1ff8d93a1ae344ca\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5e94a857566b7a589397255f076e33ec11cfbb5559c02fd0192d475be685f07b\\\"],\\\"sizeBytes\\\":268695823},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:57b61ec1af2b592c8217f770864d5d3ffb8f77f17e1922158c3fab15728d6aa3\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bb900adbd42cce67de7a3794fc8df627bffa5c23dd6510a13d7da92bbb7e596b\\\"],\\\"sizeBytes\\\":261192974},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:aa590e7fc5391b1007aa63e4e4488de68ce3695cc319d50a3f9b438deda90b29\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e054502a06509b4e69c48c890fe1cfceef89af1e78337d1a64f9dcf87a672755\\\"],\\\"sizeBytes\\\":260934429},{\\\"names\\\":[\\\"quay.io/openshift/origin-insights-runtime-exporter@sha256:4361ca1dc004b38547adeb65e35e05338ab7ab119e3d501838bc94a20acf7e02\\\",\\\"quay.io/openshift/origin-insights-runtime-exporter@sha256:f955508b8f36095ebd2e9fdfed56acbf1712ede85970571261addbe9ec618927\\\",\\\"quay.io/openshift/origin-insights-runtime-exporter:latest\\\"],\\\"sizeBytes\\\":257663969},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c14e54fa5b5449e7d7953eee6abbf144dfcbf02d70aeccaba86d89807a90703c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cc8512072d0aab350bec11d8eaf1b78d3f5ad0dc465c18644f653ae4778f59c6\\\"],\\\"sizeBytes\\\":250725761},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0ff82167f788e3fcf55ee66ba1deaaf18d0c8762020e2dea5e8e6c1e705dbb22\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73de210988add57998b2362d226fd6093b3b80c6ae31e2987402f4fecab193b8\\\"],\\\"sizeBytes\\\":248403982},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2418cd42270deeac1e44d28fbe1565fae655717906932301393ff532665398c9\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:851f636b578409323fcb5f2abdfdbcf238484c22efb4046849550daa4360900f\\\"],\\\"sizeBytes\\\":244001663},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cb19c2b6274f2994c0e353aeef1cc63a155e24dedc7349bdaa2a724eb2df598d\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5\\\"],\\\"sizeBytes\\\":232077543},{\\\"names\\\":[\\\"registry.redhat.io/leader-worker-set/lws-rhel9-operator@sha256:c202bfa15626262ff22682b64ac57539d28dd35f5960c490f5afea75cef34309\\\",\\\"registry.redhat.io/leader-worker-set/lws-rhel9-operator@sha256:e454f0147070fc62779834d1fe721eadc038d8aa746cd37ec328d6d60d485c78\\\"],\\\"sizeBytes\\\":215770828},{\\\"names\\\":[\\\"registry.redhat.io/leader-worker-set/lws-rhel9@sha256:70099611ea7781bb7b113af98e8a85ddf0ab01d5fe804cf4fec44d66a569940f\\\",\\\"registry.redhat.io/leader-worker-set/lws-rhel9@sha256:affb303b1173c273231bb50fef07310b0e220d2f08bfc0aa5912d0825e3e0d4f\\\"],\\\"sizeBytes\\\":181422385},{\\\"names\\\":[\\\"registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9\\\",\\\"registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0cb40c6d9a9e981f055d77ede0b397f00b03e44fa2c0a188c49aff65abf89a43\\\"],\\\"sizeBytes\\\":181034515},{\\\"names\\\":[\\\"quay.io/kuadrant/limitador-operator@sha256:3707fe24c6052d5adf240e12d34110706f0ac0877373830b67daab7c9a5c380d\\\",\\\"quay.io/kuadrant/limitador-operator@sha256:e19a7278fe117b1e5b57f8f3052697e354ce31ccfd7399f3e174afe66df317a0\\\",\\\"quay.io/kuadrant/limitador-operator:v0.17.1\\\"],\\\"sizeBytes\\\":68526415}]}}\" for node \"ip-10-0-131-230.ec2.internal\": Patch \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-131-230.ec2.internal/status?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 06 19:10:44.147796 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:44.147751 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" podUID="b2d540f2-9dec-4d75-842d-815689156ee0" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.30:8081/readyz\": dial tcp 10.134.0.30:8081: connect: connection refused" Jul 06 19:10:44.583453 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:44.583402 2557 generic.go:356] "Generic (PLEG): container finished" podID="b2d540f2-9dec-4d75-842d-815689156ee0" containerID="0f3252e4a72a62b3859791f134fc8d2006f30019b0d3512331a89f885b4fb9a7" exitCode=1 Jul 06 19:10:44.583645 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:44.583476 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" event={"ID":"b2d540f2-9dec-4d75-842d-815689156ee0","Type":"ContainerDied","Data":"0f3252e4a72a62b3859791f134fc8d2006f30019b0d3512331a89f885b4fb9a7"} Jul 06 19:10:44.583645 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:44.583543 2557 scope.go:117] "RemoveContainer" containerID="cf28b03bb6d11ca6b332bb06db10f418d675e1181d31ea791e7851c9be0b65bd" Jul 06 19:10:44.583885 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:44.583863 2557 scope.go:117] "RemoveContainer" containerID="0f3252e4a72a62b3859791f134fc8d2006f30019b0d3512331a89f885b4fb9a7" Jul 06 19:10:44.584106 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:10:44.584078 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=limitador-operator-controller-manager-859d7b78bd-gd8lr_kuadrant-system(b2d540f2-9dec-4d75-842d-815689156ee0)\"" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" podUID="b2d540f2-9dec-4d75-842d-815689156ee0" Jul 06 19:10:44.585402 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:44.585379 2557 generic.go:356] "Generic (PLEG): container finished" podID="61b53c91-e1dc-4f9f-942c-e9e6612cc4c7" containerID="8f679c1b05720d38c60f3f9dd2fd531d7f2a6291b08baacea0839fa6a30595b2" exitCode=1 Jul 06 19:10:44.585524 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:44.585408 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" event={"ID":"61b53c91-e1dc-4f9f-942c-e9e6612cc4c7","Type":"ContainerDied","Data":"8f679c1b05720d38c60f3f9dd2fd531d7f2a6291b08baacea0839fa6a30595b2"} Jul 06 19:10:44.585722 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:44.585709 2557 scope.go:117] "RemoveContainer" containerID="8f679c1b05720d38c60f3f9dd2fd531d7f2a6291b08baacea0839fa6a30595b2" Jul 06 19:10:44.668016 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:44.667967 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:10:45.593228 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:45.593183 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" event={"ID":"61b53c91-e1dc-4f9f-942c-e9e6612cc4c7","Type":"ContainerStarted","Data":"8a49ac1b3584213edc4f2a4a988fa906d46513950100e15f5f20d50941e3cfb5"} Jul 06 19:10:45.593621 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:45.593314 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:10:49.484178 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:10:49.484119 2557 controller.go:195] "Failed to update lease" err="Put \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-131-230.ec2.internal?timeout=10s\": context deadline exceeded" Jul 06 19:10:54.147954 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:54.147901 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:10:54.148369 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:54.148248 2557 scope.go:117] "RemoveContainer" containerID="0f3252e4a72a62b3859791f134fc8d2006f30019b0d3512331a89f885b4fb9a7" Jul 06 19:10:54.189622 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:10:54.189470 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:10:44Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:10:44Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:10:44Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:10:44Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a70169bf82777e89a725ee06c18161a796d50f5dc09d3b1980d8cf15b5c48b7b\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b90fb00a6f277d9e99d519aaa9cd2efb6f3d1c8b48f444d5135c733346c76b90\\\"],\\\"sizeBytes\\\":2092264135},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:511f5069a16301bd6df533513412d0ca308fcad1edf5859bc53cbc973f17d6df\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\"],\\\"sizeBytes\\\":1438655886},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:55ec7147f859301bbb8288966c8a21c5db729200f3c3d91db671978baef79517\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d87af67c8f369963bc01220c5cf1d56af737b9bcd46947398a3128b7137e1d1a\\\"],\\\"sizeBytes\\\":1208832350},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1537aadb2d8808accf423891ceb3a6a2deb0111bc02666f0c8df5591d7abc92\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f5c00c67012fedc41b1c472dbaa9d6dece63c9a82747da62b3243fa91cad3c35\\\"],\\\"sizeBytes\\\":901961931},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5be8cdcf34f35a89a70b6bfa49e66897641871237570b3e09766f83a0a473e6a\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5dd08f95712ac6420fe74f5fe5689a11a8b4114a22e0d84dc2d76eed6bd4f4fe\\\"],\\\"sizeBytes\\\":612408602},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1cbdec18368e5cf46bfda529c7e8a8ad32261b64d0b421ec69da4a3b9868751d\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2d26421a926916d11490ce5dbff564dad6e8978a5e5fa392a63b25beca2848aa\\\"],\\\"sizeBytes\\\":606240501},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2d5a37efb4a917780398d375b7156193497f4554ae216ef110a445724d665111\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:af46126bb56c6e82bf5610e51889b99320ba681c23ea776c7533c233f2a3c5b7\\\"],\\\"sizeBytes\\\":601507442},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c578e9a2a8e1e9b1c1adfea042f3a559baf1dcc0a601d47f2ab2360af87e7a74\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1d4a9e325ceca01359ab5b6ec7f7dcac51f1c76b1721d214aec0e01820ab592\\\"],\\\"sizeBytes\\\":563573142},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3901dd6834e7bed4942e1e4ceb7f4817615d534ccdd274aba200bbc587183585\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0f6241919ad7cdf81e4b35775946f82ea8fa362d39096a34b3ae3de7c003904\\\"],\\\"sizeBytes\\\":500548708},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0dc1d2bf1c7b77613eb12dc3dec7323cfde9a04673ea7ed31edf4da775c22642\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a9b1c2b280c23e90adf1d0d669ca39e0d6829205fbe27e725afe5c35e0037ca6\\\"],\\\"sizeBytes\\\":482011914},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6082ff851706c560884555c640798ce7a028ddf6b3a3cfa2f7579323f858a826\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:da9a648c75766e888f7d5037d75344f75b1d319d011bc5217e0d3ba3d5071a88\\\"],\\\"sizeBytes\\\":463915795},{\\\"names\\\":[\\\"quay.io/openshift/origin-insights-runtime-extractor@sha256:c4f7b0c6eefe8b18e6d8a4bc11b3b70f951f811dd551b392ebf235730a8c1e97\\\",\\\"quay.io/openshift/origin-insights-runtime-extractor@sha256:d6a4414d94942241301f29743a489515ee3bf951d9277de040d20882cd45204f\\\",\\\"quay.io/openshift/origin-insights-runtime-extractor:latest\\\"],\\\"sizeBytes\\\":421414665},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4e326414ebdae7f05b02e7778707689b9cd9638fa71e5737357fc43b790c9bdd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d3e33a8c1a0a582f1e62f55562b0da406d0675b6f677e046cff78401afcb7544\\\"],\\\"sizeBytes\\\":412532612},{\\\"names\\\":[\\\"registry.redhat.io/rhel9/postgresql-16@sha256:1149c78dbc442a419d6d9073dd5b60dd40d5639070b8cd8cd8dfab30ef88a913\\\",\\\"registry.redhat.io/rhel9/postgresql-16@sha256:cb66eb4b438ce3f9db194dd1948ce25dc4142e789025f214324b7baa3c915c05\\\",\\\"registry.redhat.io/rhel9/postgresql-16:latest\\\"],\\\"sizeBytes\\\":403303517},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:51c853fa1a08088e44a396ed8321c824b483bab3673f8a060ba5732a88deb565\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8cde2dd29b991fab261b34481407faace228f79f1e92bdeef6c97eec1af7538e\\\"],\\\"sizeBytes\\\":375179023},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:28389dd118629aa9d7c2525202f3ad6359afb98b6a3bf7672cb1ecfae2418e23\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:93d5659f2d5779d0df45117065725d0184040149328a4ce4b5e59bb2d06e230a\\\"],\\\"sizeBytes\\\":372388955},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1152d0088e99aeef75203e7e9185507833379e9829161a9f78af5500ca944f46\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:49287a1683e255a8c7d468520a3bd9e589ec8c38863ecc28ce4886ed0cce9828\\\"],\\\"sizeBytes\\\":368851365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:81fb18e5930816a580cbd2c3efe355f747b5d7341242637e44fa7182ee079800\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c1b749066725367c3e651210210d90a9e389d1f4ba4c20431ab4cab5001b31a0\\\"],\\\"sizeBytes\\\":350735931},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:022f1a802b21c79f11da33266bd78688e33f8fa5cd49189e8314ad987e1046bc\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:437a5fd01dd4bfbdd635d5069df3ee7ee9095636618f521f8c8f35dc792e75ca\\\"],\\\"sizeBytes\\\":343923425},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d6df0cb6282bc5ba571a8de322fe598afdf22e817f5a7dc92db3274dbdb9a1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:534df4b689087b5ab447fe3e97601dfb9617ea784c05dfbbde5e6a3ebc15e929\\\"],\\\"sizeBytes\\\":342879910},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1c2a3f6980952094d9116ab4ae3fbff42dc54c9a4ee9a031c31ef6b4d9d5cb9b\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:60d76d51dac424f46a693b8b319b38803b48ebb091938c4153ffa0fafb50fd11\\\"],\\\"sizeBytes\\\":338739808},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6881867b07165b6f1d0161f646d6d4ebfe2d9979327b482e6f8df0fd4c412cbe\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d95a1c6f868ae4337487acd1b7239a8f40c2905b7574570d66655a2caf268ad6\\\"],\\\"sizeBytes\\\":325729914},{\\\"names\\\":[\\\"quay.io/openshift/origin-kube-rbac-proxy@sha256:9fa7c0125c5217797dcd586e7cc44e9c1d6a8f931bcb4d3eddaa2d1b0c27e2ce\\\",\\\"quay.io/openshift/origin-kube-rbac-proxy@sha256:d82c862da8fe5446edc0eb8fbed1f1f6bc38aac7d7c1586cad0e03d3efd6f90e\\\",\\\"quay.io/openshift/origin-kube-rbac-proxy:latest\\\"],\\\"sizeBytes\\\":325671898},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8e3d4671bf7f2003bfb95459b9e5e3167d51966c587c887b63c067e9a6001d09\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eb2912aa612e75e08046f10aa8408f79e2578917c8ce1f17ec0772a2bd4f02a2\\\"],\\\"sizeBytes\\\":314111041},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:00aaa47e1b08f63b4600d9c19fd0b7f3d100d4b022fd98493aebf85d831b3693\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b2e954d9267f62174ff45a44976fa7a47e58d87fef14ffbb9fc0f8184e722280\\\"],\\\"sizeBytes\\\":309676442},{\\\"names\\\":[\\\"registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:681a5f3e02131df294a2b83933584403fe4035d43d98530a13f83a77f71171fa\\\",\\\"registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:f3683d29aa703829fb0d0ba93711188fb717cc1ac156877c9d0ab5fb5083513f\\\"],\\\"sizeBytes\\\":308936624},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:599e9f399a5256df7848dc86f5a7c14748dbeff49d3644fd7935489d07072e0e\\\"],\\\"sizeBytes\\\":303912755},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:919cd188941de1705b0b309755fd19c065999938b4c561ad08e8476ea9116f44\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6023b6bce522f459b4e662ae637701870f6296d329e13d27e05f602cbb56096\\\"],\\\"sizeBytes\\\":300938814},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:038e50a65e9e4447cb568b8672318f02bbd9376d10c835bb0e01a3126721d8bd\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fa6250524898eaddbd8f18dcc345df6d71238b0ebc4e0598485057ef7aee2f67\\\"],\\\"sizeBytes\\\":295995367},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:103b203392b4f83ec0554d1bc91ddea428aa328719707803ce8551a1d1aabd32\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eb5c1bc090355919da7a3f8a1f4f3f66244e61a0d7ac927c545faa373ead6905\\\"],\\\"sizeBytes\\\":293810026},{\\\"names\\\":[\\\"quay.io/opendatahub/opendatahub-operator@sha256:62da7505a0bd1ed943c21ede46ffb0281020eba3b56657307e73ccf96a4ee006\\\",\\\"quay.io/opendatahub/opendatahub-operator@sha256:69b729c6ba9c3d7f1f9c604d5d6f10079c9997224552da5dca3186e90f7512fd\\\",\\\"quay.io/opendatahub/opendatahub-operator:v3.5.0-ea.2\\\"],\\\"sizeBytes\\\":293029345},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:25d3a4a674f67b96b494f454f4a0bbc2b242afd1990d734e821a905ea1b6b429\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5079f558a4811d9761daaf08dd2bff84fbfe9ac5ae6a91ed8ea7c8f899605427\\\"],\\\"sizeBytes\\\":292347304},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:773bab9410669d654b42e5d3cceca5989c5be7652ef9f6ff72179ae0c63e0dac\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d6927f980baf4af326e1a723be59b315894ba986ea4fb5e9fe9cb045575dc2b7\\\"],\\\"sizeBytes\\\":276586373},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:121de0f318d50621c54c9ceedd72d975745cdef8adbfce041113cd3ef3543090\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:54daab5a81cfe8c31382d27a41095dcd8db35bc07acdc90f3cede90ffc06a782\\\"],\\\"sizeBytes\\\":275908478},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e7040afaaa68ee77d97ff176987a40ae096a2808c0b732796fd9512e980384d7\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fff38eefce4202e98d3f93d274a73c576403ab43d85609f72f414922ada6cf50\\\"],\\\"sizeBytes\\\":274511508},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:61ec103ea7e6351f76851bc82db635db95646d2528da39cf6093bce74d33338c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6aa8e24edbb4f884870dd2b5189e19d18884586cb62f391f1dd32891283482d4\\\"],\\\"sizeBytes\\\":273315827},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d77ed4b5716efb9fea719ee357ea26d99436291ac7e648c1ff8d93a1ae344ca\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5e94a857566b7a589397255f076e33ec11cfbb5559c02fd0192d475be685f07b\\\"],\\\"sizeBytes\\\":268695823},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:57b61ec1af2b592c8217f770864d5d3ffb8f77f17e1922158c3fab15728d6aa3\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bb900adbd42cce67de7a3794fc8df627bffa5c23dd6510a13d7da92bbb7e596b\\\"],\\\"sizeBytes\\\":261192974},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:aa590e7fc5391b1007aa63e4e4488de68ce3695cc319d50a3f9b438deda90b29\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e054502a06509b4e69c48c890fe1cfceef89af1e78337d1a64f9dcf87a672755\\\"],\\\"sizeBytes\\\":260934429},{\\\"names\\\":[\\\"quay.io/openshift/origin-insights-runtime-exporter@sha256:4361ca1dc004b38547adeb65e35e05338ab7ab119e3d501838bc94a20acf7e02\\\",\\\"quay.io/openshift/origin-insights-runtime-exporter@sha256:f955508b8f36095ebd2e9fdfed56acbf1712ede85970571261addbe9ec618927\\\",\\\"quay.io/openshift/origin-insights-runtime-exporter:latest\\\"],\\\"sizeBytes\\\":257663969},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c14e54fa5b5449e7d7953eee6abbf144dfcbf02d70aeccaba86d89807a90703c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cc8512072d0aab350bec11d8eaf1b78d3f5ad0dc465c18644f653ae4778f59c6\\\"],\\\"sizeBytes\\\":250725761},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0ff82167f788e3fcf55ee66ba1deaaf18d0c8762020e2dea5e8e6c1e705dbb22\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:73de210988add57998b2362d226fd6093b3b80c6ae31e2987402f4fecab193b8\\\"],\\\"sizeBytes\\\":248403982},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2418cd42270deeac1e44d28fbe1565fae655717906932301393ff532665398c9\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:851f636b578409323fcb5f2abdfdbcf238484c22efb4046849550daa4360900f\\\"],\\\"sizeBytes\\\":244001663},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cb19c2b6274f2994c0e353aeef1cc63a155e24dedc7349bdaa2a724eb2df598d\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5\\\"],\\\"sizeBytes\\\":232077543},{\\\"names\\\":[\\\"registry.redhat.io/leader-worker-set/lws-rhel9-operator@sha256:c202bfa15626262ff22682b64ac57539d28dd35f5960c490f5afea75cef34309\\\",\\\"registry.redhat.io/leader-worker-set/lws-rhel9-operator@sha256:e454f0147070fc62779834d1fe721eadc038d8aa746cd37ec328d6d60d485c78\\\"],\\\"sizeBytes\\\":215770828},{\\\"names\\\":[\\\"registry.redhat.io/leader-worker-set/lws-rhel9@sha256:70099611ea7781bb7b113af98e8a85ddf0ab01d5fe804cf4fec44d66a569940f\\\",\\\"registry.redhat.io/leader-worker-set/lws-rhel9@sha256:affb303b1173c273231bb50fef07310b0e220d2f08bfc0aa5912d0825e3e0d4f\\\"],\\\"sizeBytes\\\":181422385},{\\\"names\\\":[\\\"registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9\\\",\\\"registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0cb40c6d9a9e981f055d77ede0b397f00b03e44fa2c0a188c49aff65abf89a43\\\"],\\\"sizeBytes\\\":181034515},{\\\"names\\\":[\\\"quay.io/kuadrant/limitador-operator@sha256:3707fe24c6052d5adf240e12d34110706f0ac0877373830b67daab7c9a5c380d\\\",\\\"quay.io/kuadrant/limitador-operator@sha256:e19a7278fe117b1e5b57f8f3052697e354ce31ccfd7399f3e174afe66df317a0\\\",\\\"quay.io/kuadrant/limitador-operator:v0.17.1\\\"],\\\"sizeBytes\\\":68526415}]}}\" for node \"ip-10-0-131-230.ec2.internal\": Patch \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-131-230.ec2.internal/status?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 06 19:10:54.636389 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:54.636353 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" event={"ID":"b2d540f2-9dec-4d75-842d-815689156ee0","Type":"ContainerStarted","Data":"67021d0a4f03764c7966a7a6e65567a682fb5c5e67f3a5fdd9cb74f7426604da"} Jul 06 19:10:54.636604 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:54.636587 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:10:56.600603 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:10:56.600566 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-87xf5" Jul 06 19:10:59.484800 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:10:59.484756 2557 controller.go:195] "Failed to update lease" err="Put \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-131-230.ec2.internal?timeout=10s\": context deadline exceeded" Jul 06 19:11:04.190315 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:11:04.190268 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-230.ec2.internal\": Get \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-131-230.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 06 19:11:04.688368 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:04.688333 2557 generic.go:356] "Generic (PLEG): container finished" podID="b2d540f2-9dec-4d75-842d-815689156ee0" containerID="67021d0a4f03764c7966a7a6e65567a682fb5c5e67f3a5fdd9cb74f7426604da" exitCode=1 Jul 06 19:11:04.688593 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:04.688405 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" event={"ID":"b2d540f2-9dec-4d75-842d-815689156ee0","Type":"ContainerDied","Data":"67021d0a4f03764c7966a7a6e65567a682fb5c5e67f3a5fdd9cb74f7426604da"} Jul 06 19:11:04.688593 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:04.688478 2557 scope.go:117] "RemoveContainer" containerID="0f3252e4a72a62b3859791f134fc8d2006f30019b0d3512331a89f885b4fb9a7" Jul 06 19:11:04.688830 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:04.688814 2557 scope.go:117] "RemoveContainer" containerID="67021d0a4f03764c7966a7a6e65567a682fb5c5e67f3a5fdd9cb74f7426604da" Jul 06 19:11:04.689082 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:11:04.689059 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=limitador-operator-controller-manager-859d7b78bd-gd8lr_kuadrant-system(b2d540f2-9dec-4d75-842d-815689156ee0)\"" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" podUID="b2d540f2-9dec-4d75-842d-815689156ee0" Jul 06 19:11:09.485571 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:11:09.485524 2557 controller.go:195] "Failed to update lease" err="Put \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-131-230.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 06 19:11:14.191322 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:11:14.191226 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-230.ec2.internal\": Get \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-131-230.ec2.internal?timeout=10s\": context deadline exceeded" Jul 06 19:11:17.774886 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:17.774849 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:11:17.775289 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:17.775210 2557 scope.go:117] "RemoveContainer" containerID="67021d0a4f03764c7966a7a6e65567a682fb5c5e67f3a5fdd9cb74f7426604da" Jul 06 19:11:17.775420 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:11:17.775399 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=limitador-operator-controller-manager-859d7b78bd-gd8lr_kuadrant-system(b2d540f2-9dec-4d75-842d-815689156ee0)\"" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" podUID="b2d540f2-9dec-4d75-842d-815689156ee0" Jul 06 19:11:19.486816 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:11:19.486773 2557 controller.go:195] "Failed to update lease" err="Put \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-131-230.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 06 19:11:28.190841 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:28.190806 2557 scope.go:117] "RemoveContainer" containerID="67021d0a4f03764c7966a7a6e65567a682fb5c5e67f3a5fdd9cb74f7426604da" Jul 06 19:11:28.813366 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:28.813336 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" event={"ID":"b2d540f2-9dec-4d75-842d-815689156ee0","Type":"ContainerStarted","Data":"c91e6c251532eff60791ce3680462085f773a1b8cc414082c14c783cef98c7d6"} Jul 06 19:11:28.813607 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:28.813577 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:11:37.836278 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:37.836216 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:39.820402 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:39.820373 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:11:46.991491 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:46.991454 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:49.114664 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:49.114623 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:53.048222 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:53.048188 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:53.306855 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:11:53.306754 2557 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"b2d540f2-9dec-4d75-842d-815689156ee0\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-06T19:10:44Z\\\",\\\"message\\\":\\\"containers with unready status: [manager]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-06T19:10:44Z\\\",\\\"message\\\":\\\"containers with unready status: [manager]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"200m\\\",\\\"memory\\\":\\\"200Mi\\\"},\\\"containerID\\\":\\\"cri-o://c91e6c251532eff60791ce3680462085f773a1b8cc414082c14c783cef98c7d6\\\",\\\"image\\\":\\\"quay.io/kuadrant/limitador-operator:v0.17.1\\\",\\\"imageID\\\":\\\"quay.io/kuadrant/limitador-operator@sha256:3707fe24c6052d5adf240e12d34110706f0ac0877373830b67daab7c9a5c380d\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://67021d0a4f03764c7966a7a6e65567a682fb5c5e67f3a5fdd9cb74f7426604da\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-06T19:11:04Z\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-06T19:10:54Z\\\"}},\\\"name\\\":\\\"manager\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"200m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"200m\\\",\\\"memory\\\":\\\"200Mi\\\"}},\\\"restartCount\\\":3,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-06T19:11:28Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000680000],\\\"uid\\\":1000680000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-ccdcr\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-gd8lr\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ed6006ff-308f-4519-b84f-e305d9418cee.svc:9743/pod?timeout=10s\": context deadline exceeded" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:11:55.090682 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:55.090605 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:55.363367 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:55.363274 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:58.201684 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:11:58.201634 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:11:48Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:11:48Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:11:48Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:11:48Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-131-230.ec2.internal\": Patch \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-131-230.ec2.internal/status?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 06 19:11:58.253520 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:11:58.253484 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:01.874456 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:01.874406 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:03.348481 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:12:03.348412 2557 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"b2d540f2-9dec-4d75-842d-815689156ee0\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-06T19:11:39Z\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-06T19:11:39Z\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"200m\\\",\\\"memory\\\":\\\"200Mi\\\"},\\\"containerID\\\":\\\"cri-o://c91e6c251532eff60791ce3680462085f773a1b8cc414082c14c783cef98c7d6\\\",\\\"image\\\":\\\"quay.io/kuadrant/limitador-operator:v0.17.1\\\",\\\"imageID\\\":\\\"quay.io/kuadrant/limitador-operator@sha256:3707fe24c6052d5adf240e12d34110706f0ac0877373830b67daab7c9a5c380d\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://67021d0a4f03764c7966a7a6e65567a682fb5c5e67f3a5fdd9cb74f7426604da\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-06T19:11:04Z\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-06T19:10:54Z\\\"}},\\\"name\\\":\\\"manager\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"200m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"200m\\\",\\\"memory\\\":\\\"200Mi\\\"}},\\\"restartCount\\\":3,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-06T19:11:28Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000680000],\\\"uid\\\":1000680000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-ccdcr\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-gd8lr\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ed6006ff-308f-4519-b84f-e305d9418cee.svc:9743/pod?timeout=10s\": context deadline exceeded" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:12:10.611387 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:10.611346 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:12.512991 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:12.512958 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:30.111899 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:30.111862 2557 generic.go:356] "Generic (PLEG): container finished" podID="b2d540f2-9dec-4d75-842d-815689156ee0" containerID="c91e6c251532eff60791ce3680462085f773a1b8cc414082c14c783cef98c7d6" exitCode=1 Jul 06 19:12:30.112415 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:30.111946 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" event={"ID":"b2d540f2-9dec-4d75-842d-815689156ee0","Type":"ContainerDied","Data":"c91e6c251532eff60791ce3680462085f773a1b8cc414082c14c783cef98c7d6"} Jul 06 19:12:30.112415 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:30.112012 2557 scope.go:117] "RemoveContainer" containerID="67021d0a4f03764c7966a7a6e65567a682fb5c5e67f3a5fdd9cb74f7426604da" Jul 06 19:12:30.112415 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:30.112333 2557 scope.go:117] "RemoveContainer" containerID="c91e6c251532eff60791ce3680462085f773a1b8cc414082c14c783cef98c7d6" Jul 06 19:12:30.112599 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:12:30.112580 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=limitador-operator-controller-manager-859d7b78bd-gd8lr_kuadrant-system(b2d540f2-9dec-4d75-842d-815689156ee0)\"" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" podUID="b2d540f2-9dec-4d75-842d-815689156ee0" Jul 06 19:12:30.114170 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:30.114150 2557 generic.go:356] "Generic (PLEG): container finished" podID="c2780119-d899-424a-895e-9ef16b4c3cab" containerID="e2449e1a86ed382152aaef0fbbf092cf5836dc7559c2b2ae4a22ea0cbf458102" exitCode=1 Jul 06 19:12:30.114257 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:30.114192 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" event={"ID":"c2780119-d899-424a-895e-9ef16b4c3cab","Type":"ContainerDied","Data":"e2449e1a86ed382152aaef0fbbf092cf5836dc7559c2b2ae4a22ea0cbf458102"} Jul 06 19:12:30.114557 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:30.114543 2557 scope.go:117] "RemoveContainer" containerID="e2449e1a86ed382152aaef0fbbf092cf5836dc7559c2b2ae4a22ea0cbf458102" Jul 06 19:12:31.122846 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:31.122809 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" event={"ID":"c2780119-d899-424a-895e-9ef16b4c3cab","Type":"ContainerStarted","Data":"eba3184ab2762964c343217feec3347cd247d32973d4c05f0c612d91de323d2c"} Jul 06 19:12:31.123263 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:31.123035 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:12:31.725957 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:31.725913 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:31.982190 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:31.982098 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:32.209163 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:32.209130 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:36.246152 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:36.246116 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:37.775088 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:37.775052 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:12:37.775509 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:37.775433 2557 scope.go:117] "RemoveContainer" containerID="c91e6c251532eff60791ce3680462085f773a1b8cc414082c14c783cef98c7d6" Jul 06 19:12:37.775701 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:12:37.775679 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=limitador-operator-controller-manager-859d7b78bd-gd8lr_kuadrant-system(b2d540f2-9dec-4d75-842d-815689156ee0)\"" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" podUID="b2d540f2-9dec-4d75-842d-815689156ee0" Jul 06 19:12:38.137180 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:38.137093 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6dcbd56464-f8dd6"] Jul 06 19:12:38.140897 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:38.140880 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dcbd56464-f8dd6" Jul 06 19:12:38.153291 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:38.153272 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 19:12:38.163543 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:38.163520 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 19:12:38.164140 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:38.164111 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:38.245844 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:38.245807 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6dcbd56464-f8dd6"] Jul 06 19:12:38.298304 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:38.298271 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95rv9\" (UniqueName: \"kubernetes.io/projected/097546ed-0beb-4061-811a-b5a6c7b1254e-kube-api-access-95rv9\") pod \"maas-controller-6dcbd56464-f8dd6\" (UID: \"097546ed-0beb-4061-811a-b5a6c7b1254e\") " pod="opendatahub/maas-controller-6dcbd56464-f8dd6" Jul 06 19:12:38.298490 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:38.298355 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/097546ed-0beb-4061-811a-b5a6c7b1254e-webhook-cert\") pod \"maas-controller-6dcbd56464-f8dd6\" (UID: \"097546ed-0beb-4061-811a-b5a6c7b1254e\") " pod="opendatahub/maas-controller-6dcbd56464-f8dd6" Jul 06 19:12:38.399595 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:38.399517 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/097546ed-0beb-4061-811a-b5a6c7b1254e-webhook-cert\") pod \"maas-controller-6dcbd56464-f8dd6\" (UID: \"097546ed-0beb-4061-811a-b5a6c7b1254e\") " pod="opendatahub/maas-controller-6dcbd56464-f8dd6" Jul 06 19:12:38.399735 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:38.399608 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95rv9\" (UniqueName: \"kubernetes.io/projected/097546ed-0beb-4061-811a-b5a6c7b1254e-kube-api-access-95rv9\") pod \"maas-controller-6dcbd56464-f8dd6\" (UID: \"097546ed-0beb-4061-811a-b5a6c7b1254e\") " pod="opendatahub/maas-controller-6dcbd56464-f8dd6" Jul 06 19:12:38.401955 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:38.401924 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/097546ed-0beb-4061-811a-b5a6c7b1254e-webhook-cert\") pod \"maas-controller-6dcbd56464-f8dd6\" (UID: \"097546ed-0beb-4061-811a-b5a6c7b1254e\") " pod="opendatahub/maas-controller-6dcbd56464-f8dd6" Jul 06 19:12:38.426691 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:38.426665 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95rv9\" (UniqueName: \"kubernetes.io/projected/097546ed-0beb-4061-811a-b5a6c7b1254e-kube-api-access-95rv9\") pod \"maas-controller-6dcbd56464-f8dd6\" (UID: \"097546ed-0beb-4061-811a-b5a6c7b1254e\") " pod="opendatahub/maas-controller-6dcbd56464-f8dd6" Jul 06 19:12:38.451657 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:38.451615 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dcbd56464-f8dd6" Jul 06 19:12:38.479941 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:38.479909 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:38.681782 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:38.681712 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6dcbd56464-f8dd6"] Jul 06 19:12:38.684448 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:12:38.684395 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod097546ed_0beb_4061_811a_b5a6c7b1254e.slice/crio-79b1caca9a0ffc130e4a5e3ebe67f0bddbbc0c5c5da215cebb746b1cf6c54eec WatchSource:0}: Error finding container 79b1caca9a0ffc130e4a5e3ebe67f0bddbbc0c5c5da215cebb746b1cf6c54eec: Status 404 returned error can't find the container with id 79b1caca9a0ffc130e4a5e3ebe67f0bddbbc0c5c5da215cebb746b1cf6c54eec Jul 06 19:12:38.843649 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:38.843615 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:39.168624 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:39.168585 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dcbd56464-f8dd6" event={"ID":"097546ed-0beb-4061-811a-b5a6c7b1254e","Type":"ContainerStarted","Data":"79b1caca9a0ffc130e4a5e3ebe67f0bddbbc0c5c5da215cebb746b1cf6c54eec"} Jul 06 19:12:39.819539 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:39.819500 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:12:39.819973 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:39.819948 2557 scope.go:117] "RemoveContainer" containerID="c91e6c251532eff60791ce3680462085f773a1b8cc414082c14c783cef98c7d6" Jul 06 19:12:39.820244 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:12:39.820214 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=limitador-operator-controller-manager-859d7b78bd-gd8lr_kuadrant-system(b2d540f2-9dec-4d75-842d-815689156ee0)\"" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" podUID="b2d540f2-9dec-4d75-842d-815689156ee0" Jul 06 19:12:41.089964 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:41.089920 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:41.862243 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:41.862137 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:42.130719 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:42.130636 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m" Jul 06 19:12:42.187270 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:42.187199 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dcbd56464-f8dd6" event={"ID":"097546ed-0beb-4061-811a-b5a6c7b1254e","Type":"ContainerStarted","Data":"f2a675247def7a53e63027fea53b648ff6870e8d486230aec6758c44e0239427"} Jul 06 19:12:42.187462 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:42.187298 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6dcbd56464-f8dd6" Jul 06 19:12:42.275120 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:42.275070 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6dcbd56464-f8dd6" podStartSLOduration=1.4394676020000001 podStartE2EDuration="4.275052689s" totalImagesPullingTime="2.835585087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:12:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:12:42.27458616 +0000 UTC m=+802.635749951" watchObservedRunningTime="2026-07-06 19:12:42.275052689 +0000 UTC m=+802.636216481" Jul 06 19:12:42.327745 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:42.327711 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:43.488135 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:43.488099 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:45.391363 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:45.391323 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:45.779078 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:45.779041 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:46.255422 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:46.255365 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:46.420075 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:46.420036 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:47.028751 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:47.028708 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:47.523520 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:47.523490 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:47.819571 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:47.819469 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:49.773897 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:49.773868 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:52.191378 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:52.191348 2557 scope.go:117] "RemoveContainer" containerID="c91e6c251532eff60791ce3680462085f773a1b8cc414082c14c783cef98c7d6" Jul 06 19:12:52.191836 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:12:52.191644 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=limitador-operator-controller-manager-859d7b78bd-gd8lr_kuadrant-system(b2d540f2-9dec-4d75-842d-815689156ee0)\"" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" podUID="b2d540f2-9dec-4d75-842d-815689156ee0" Jul 06 19:12:52.913424 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:52.913386 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:53.206761 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:53.206673 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6dcbd56464-f8dd6" Jul 06 19:12:53.659499 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:53.659459 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6dcbd56464-f8dd6"] Jul 06 19:12:53.659759 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:53.659709 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6dcbd56464-f8dd6" podUID="097546ed-0beb-4061-811a-b5a6c7b1254e" containerName="manager" containerID="cri-o://f2a675247def7a53e63027fea53b648ff6870e8d486230aec6758c44e0239427" gracePeriod=30 Jul 06 19:12:53.909757 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:53.909691 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dcbd56464-f8dd6" Jul 06 19:12:53.939909 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:53.939884 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95rv9\" (UniqueName: \"kubernetes.io/projected/097546ed-0beb-4061-811a-b5a6c7b1254e-kube-api-access-95rv9\") pod \"097546ed-0beb-4061-811a-b5a6c7b1254e\" (UID: \"097546ed-0beb-4061-811a-b5a6c7b1254e\") " Jul 06 19:12:53.940074 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:53.939937 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/097546ed-0beb-4061-811a-b5a6c7b1254e-webhook-cert\") pod \"097546ed-0beb-4061-811a-b5a6c7b1254e\" (UID: \"097546ed-0beb-4061-811a-b5a6c7b1254e\") " Jul 06 19:12:53.942004 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:53.941976 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/097546ed-0beb-4061-811a-b5a6c7b1254e-kube-api-access-95rv9" (OuterVolumeSpecName: "kube-api-access-95rv9") pod "097546ed-0beb-4061-811a-b5a6c7b1254e" (UID: "097546ed-0beb-4061-811a-b5a6c7b1254e"). InnerVolumeSpecName "kube-api-access-95rv9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:12:53.942091 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:53.941981 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/097546ed-0beb-4061-811a-b5a6c7b1254e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "097546ed-0beb-4061-811a-b5a6c7b1254e" (UID: "097546ed-0beb-4061-811a-b5a6c7b1254e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:12:53.976610 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:53.976568 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:54.041375 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:54.041330 2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/097546ed-0beb-4061-811a-b5a6c7b1254e-webhook-cert\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:12:54.041375 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:54.041373 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95rv9\" (UniqueName: \"kubernetes.io/projected/097546ed-0beb-4061-811a-b5a6c7b1254e-kube-api-access-95rv9\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:12:54.247927 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:54.247890 2557 generic.go:356] "Generic (PLEG): container finished" podID="097546ed-0beb-4061-811a-b5a6c7b1254e" containerID="f2a675247def7a53e63027fea53b648ff6870e8d486230aec6758c44e0239427" exitCode=0 Jul 06 19:12:54.247927 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:54.247928 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dcbd56464-f8dd6" Jul 06 19:12:54.248417 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:54.247928 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dcbd56464-f8dd6" event={"ID":"097546ed-0beb-4061-811a-b5a6c7b1254e","Type":"ContainerDied","Data":"f2a675247def7a53e63027fea53b648ff6870e8d486230aec6758c44e0239427"} Jul 06 19:12:54.248417 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:54.247988 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dcbd56464-f8dd6" event={"ID":"097546ed-0beb-4061-811a-b5a6c7b1254e","Type":"ContainerDied","Data":"79b1caca9a0ffc130e4a5e3ebe67f0bddbbc0c5c5da215cebb746b1cf6c54eec"} Jul 06 19:12:54.248417 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:54.248015 2557 scope.go:117] "RemoveContainer" containerID="f2a675247def7a53e63027fea53b648ff6870e8d486230aec6758c44e0239427" Jul 06 19:12:54.256493 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:54.256465 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:54.257155 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:54.257138 2557 scope.go:117] "RemoveContainer" containerID="f2a675247def7a53e63027fea53b648ff6870e8d486230aec6758c44e0239427" Jul 06 19:12:54.257489 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:12:54.257423 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2a675247def7a53e63027fea53b648ff6870e8d486230aec6758c44e0239427\": container with ID starting with f2a675247def7a53e63027fea53b648ff6870e8d486230aec6758c44e0239427 not found: ID does not exist" containerID="f2a675247def7a53e63027fea53b648ff6870e8d486230aec6758c44e0239427" Jul 06 19:12:54.257596 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:54.257505 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a675247def7a53e63027fea53b648ff6870e8d486230aec6758c44e0239427"} err="failed to get container status \"f2a675247def7a53e63027fea53b648ff6870e8d486230aec6758c44e0239427\": rpc error: code = NotFound desc = could not find container \"f2a675247def7a53e63027fea53b648ff6870e8d486230aec6758c44e0239427\": container with ID starting with f2a675247def7a53e63027fea53b648ff6870e8d486230aec6758c44e0239427 not found: ID does not exist" Jul 06 19:12:54.305919 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:54.305861 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6dcbd56464-f8dd6"] Jul 06 19:12:54.311185 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:54.311157 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6dcbd56464-f8dd6"] Jul 06 19:12:56.197120 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:56.197085 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="097546ed-0beb-4061-811a-b5a6c7b1254e" path="/var/lib/kubelet/pods/097546ed-0beb-4061-811a-b5a6c7b1254e/volumes" Jul 06 19:12:57.890105 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:12:57.890068 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:00.278242 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:00.278198 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:00.594727 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:00.594630 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:03.191229 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:03.191198 2557 scope.go:117] "RemoveContainer" containerID="c91e6c251532eff60791ce3680462085f773a1b8cc414082c14c783cef98c7d6" Jul 06 19:13:03.191638 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:13:03.191385 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=limitador-operator-controller-manager-859d7b78bd-gd8lr_kuadrant-system(b2d540f2-9dec-4d75-842d-815689156ee0)\"" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" podUID="b2d540f2-9dec-4d75-842d-815689156ee0" Jul 06 19:13:07.787033 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:07.786992 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:09.925763 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:09.925723 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:13.291419 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:13.291374 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:13.836899 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:13.836864 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:13.873262 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:13.873233 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-787885975b-9sszn"] Jul 06 19:13:13.873699 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:13.873684 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="097546ed-0beb-4061-811a-b5a6c7b1254e" containerName="manager" Jul 06 19:13:13.873773 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:13.873717 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="097546ed-0beb-4061-811a-b5a6c7b1254e" containerName="manager" Jul 06 19:13:13.873811 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:13.873803 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="097546ed-0beb-4061-811a-b5a6c7b1254e" containerName="manager" Jul 06 19:13:13.878699 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:13.878684 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-787885975b-9sszn" Jul 06 19:13:13.882140 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:13.882115 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 19:13:13.923904 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:13.923872 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-787885975b-9sszn"] Jul 06 19:13:14.022234 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:14.022200 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/597c5e2d-af7f-4979-9175-d0f836161efc-openshift-service-ca\") pod \"maas-api-787885975b-9sszn\" (UID: \"597c5e2d-af7f-4979-9175-d0f836161efc\") " pod="opendatahub/maas-api-787885975b-9sszn" Jul 06 19:13:14.022421 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:14.022298 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcgvg\" (UniqueName: \"kubernetes.io/projected/597c5e2d-af7f-4979-9175-d0f836161efc-kube-api-access-vcgvg\") pod \"maas-api-787885975b-9sszn\" (UID: \"597c5e2d-af7f-4979-9175-d0f836161efc\") " pod="opendatahub/maas-api-787885975b-9sszn" Jul 06 19:13:14.022421 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:14.022342 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/597c5e2d-af7f-4979-9175-d0f836161efc-maas-api-tls\") pod \"maas-api-787885975b-9sszn\" (UID: \"597c5e2d-af7f-4979-9175-d0f836161efc\") " pod="opendatahub/maas-api-787885975b-9sszn" Jul 06 19:13:14.032855 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:14.032815 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:14.123397 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:14.123306 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcgvg\" (UniqueName: \"kubernetes.io/projected/597c5e2d-af7f-4979-9175-d0f836161efc-kube-api-access-vcgvg\") pod \"maas-api-787885975b-9sszn\" (UID: \"597c5e2d-af7f-4979-9175-d0f836161efc\") " pod="opendatahub/maas-api-787885975b-9sszn" Jul 06 19:13:14.123397 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:14.123375 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/597c5e2d-af7f-4979-9175-d0f836161efc-maas-api-tls\") pod \"maas-api-787885975b-9sszn\" (UID: \"597c5e2d-af7f-4979-9175-d0f836161efc\") " pod="opendatahub/maas-api-787885975b-9sszn" Jul 06 19:13:14.123603 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:14.123411 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/597c5e2d-af7f-4979-9175-d0f836161efc-openshift-service-ca\") pod \"maas-api-787885975b-9sszn\" (UID: \"597c5e2d-af7f-4979-9175-d0f836161efc\") " pod="opendatahub/maas-api-787885975b-9sszn" Jul 06 19:13:14.124197 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:14.124167 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/597c5e2d-af7f-4979-9175-d0f836161efc-openshift-service-ca\") pod \"maas-api-787885975b-9sszn\" (UID: \"597c5e2d-af7f-4979-9175-d0f836161efc\") " pod="opendatahub/maas-api-787885975b-9sszn" Jul 06 19:13:14.125920 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:14.125890 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/597c5e2d-af7f-4979-9175-d0f836161efc-maas-api-tls\") pod \"maas-api-787885975b-9sszn\" (UID: \"597c5e2d-af7f-4979-9175-d0f836161efc\") " pod="opendatahub/maas-api-787885975b-9sszn" Jul 06 19:13:14.146732 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:14.146702 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcgvg\" (UniqueName: \"kubernetes.io/projected/597c5e2d-af7f-4979-9175-d0f836161efc-kube-api-access-vcgvg\") pod \"maas-api-787885975b-9sszn\" (UID: \"597c5e2d-af7f-4979-9175-d0f836161efc\") " pod="opendatahub/maas-api-787885975b-9sszn" Jul 06 19:13:14.188773 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:14.188733 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-787885975b-9sszn" Jul 06 19:13:14.362398 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:14.362353 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-787885975b-9sszn"] Jul 06 19:13:14.369672 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:13:14.369632 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod597c5e2d_af7f_4979_9175_d0f836161efc.slice/crio-4d56d13be4c3334058c961ede0b6a8e8ef9e2897512aad322f4ad2db77ed73c9 WatchSource:0}: Error finding container 4d56d13be4c3334058c961ede0b6a8e8ef9e2897512aad322f4ad2db77ed73c9: Status 404 returned error can't find the container with id 4d56d13be4c3334058c961ede0b6a8e8ef9e2897512aad322f4ad2db77ed73c9 Jul 06 19:13:14.938776 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:14.938736 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:15.183703 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:15.183663 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:15.357652 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:15.357617 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-787885975b-9sszn" event={"ID":"597c5e2d-af7f-4979-9175-d0f836161efc","Type":"ContainerStarted","Data":"4d56d13be4c3334058c961ede0b6a8e8ef9e2897512aad322f4ad2db77ed73c9"} Jul 06 19:13:17.190635 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:17.190608 2557 scope.go:117] "RemoveContainer" containerID="c91e6c251532eff60791ce3680462085f773a1b8cc414082c14c783cef98c7d6" Jul 06 19:13:17.369337 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:17.369301 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-787885975b-9sszn" event={"ID":"597c5e2d-af7f-4979-9175-d0f836161efc","Type":"ContainerStarted","Data":"7922cbc51073ff6300ff479e0978281caca8c2d2d2fc2c53c11fdfaaf883b426"} Jul 06 19:13:17.369579 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:17.369466 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-787885975b-9sszn" Jul 06 19:13:17.371171 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:17.371150 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" event={"ID":"b2d540f2-9dec-4d75-842d-815689156ee0","Type":"ContainerStarted","Data":"70ef6943fbb35b8ff39f427684fb9a2e8a969a0f7e4b57cb9612f8ae91caac25"} Jul 06 19:13:17.371371 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:17.371350 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:13:17.450857 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:17.450747 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-787885975b-9sszn" podStartSLOduration=2.280328012 podStartE2EDuration="4.450731118s" totalImagesPullingTime="2.170403106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:13:17.449276579 +0000 UTC m=+837.810440372" watchObservedRunningTime="2026-07-06 19:13:17.450731118 +0000 UTC m=+837.811894909" Jul 06 19:13:18.960214 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:18.960183 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:23.380936 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:23.380903 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-787885975b-9sszn" Jul 06 19:13:28.378191 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:28.378157 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gd8lr" Jul 06 19:13:31.579642 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:31.579605 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:34.360917 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:34.360876 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:35.947660 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:35.947624 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:37.110722 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:37.110687 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:38.521667 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:38.521631 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:39.426193 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:39.426151 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:41.983129 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:41.983094 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:50.609163 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:50.609125 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:52.085996 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:52.085958 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:53.827546 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:53.827500 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:54.876155 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:54.876112 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:56.126049 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:13:56.126013 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:04.852074 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:04.852039 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:05.009288 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:05.009254 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:05.247862 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:05.247828 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:10.837416 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:10.837373 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:12.507503 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:12.507388 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:13.723689 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:13.723650 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:21.302839 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:21.302805 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:24.235155 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:24.235099 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:28.394968 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:28.394910 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:34.593470 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:34.593394 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:35.617653 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:35.617597 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:36.494978 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:36.494937 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:37.610708 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:37.610674 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:38.750289 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:38.750251 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:40.574919 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:40.574887 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:41.333716 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:41.333671 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:42.075150 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:42.075113 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:45.667838 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:45.667797 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:46.403512 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:46.403472 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:48.344513 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:48.344478 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:51.564709 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:51.564671 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:52.318574 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:52.318535 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:53.221511 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:53.221460 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:54.902086 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:54.902048 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:56.143999 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:56.143953 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:57.518045 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:57.518012 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:58.602157 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:14:58.602119 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:05.909011 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:05.908972 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:07.378343 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:07.378293 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:09.968633 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:09.968576 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:10.106717 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:10.106671 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:22.396483 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:22.396428 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:24.062881 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:24.062835 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:24.223414 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:24.223383 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:25.981515 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:25.981475 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:38.030478 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:38.030420 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:38.191654 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:38.191589 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:43.000068 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:42.999975 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:43.179781 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:43.179743 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:43.373864 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:43.373778 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:43.711111 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:43.711014 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:46.270576 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:46.270540 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:54.813104 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:54.813067 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:56.112603 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.112564 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx"] Jul 06 19:15:56.115358 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.115340 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.125536 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.125511 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 19:15:56.125699 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.125678 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 19:15:56.128690 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.128672 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 19:15:56.129157 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.129145 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kppft\"" Jul 06 19:15:56.155676 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.155642 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx"] Jul 06 19:15:56.252080 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.252040 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng7zt\" (UniqueName: \"kubernetes.io/projected/85852870-ec89-4989-a187-94520d14393e-kube-api-access-ng7zt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.252265 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.252094 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85852870-ec89-4989-a187-94520d14393e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.252265 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.252153 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85852870-ec89-4989-a187-94520d14393e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.252265 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.252182 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85852870-ec89-4989-a187-94520d14393e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.252374 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.252253 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85852870-ec89-4989-a187-94520d14393e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.252374 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.252291 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85852870-ec89-4989-a187-94520d14393e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.273389 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.273354 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:56.352925 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.352894 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85852870-ec89-4989-a187-94520d14393e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.353123 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.352940 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85852870-ec89-4989-a187-94520d14393e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.353123 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.352968 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85852870-ec89-4989-a187-94520d14393e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.353123 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.352994 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85852870-ec89-4989-a187-94520d14393e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.353123 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.353014 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85852870-ec89-4989-a187-94520d14393e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.353337 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.353149 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng7zt\" (UniqueName: \"kubernetes.io/projected/85852870-ec89-4989-a187-94520d14393e-kube-api-access-ng7zt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.353478 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.353424 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85852870-ec89-4989-a187-94520d14393e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.353630 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.353513 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85852870-ec89-4989-a187-94520d14393e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.353630 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.353615 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85852870-ec89-4989-a187-94520d14393e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.355333 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.355315 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85852870-ec89-4989-a187-94520d14393e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.355428 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.355411 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85852870-ec89-4989-a187-94520d14393e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.363287 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.363231 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng7zt\" (UniqueName: \"kubernetes.io/projected/85852870-ec89-4989-a187-94520d14393e-kube-api-access-ng7zt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wnjmx\" (UID: \"85852870-ec89-4989-a187-94520d14393e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.426268 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.426231 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:15:56.581217 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.581192 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx"] Jul 06 19:15:56.583791 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:15:56.583760 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85852870_ec89_4989_a187_94520d14393e.slice/crio-07b5f15354a0389acfe2899ae5c85063abe60a0a71cfa91666535076e85b5f19 WatchSource:0}: Error finding container 07b5f15354a0389acfe2899ae5c85063abe60a0a71cfa91666535076e85b5f19: Status 404 returned error can't find the container with id 07b5f15354a0389acfe2899ae5c85063abe60a0a71cfa91666535076e85b5f19 Jul 06 19:15:56.585150 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:56.585134 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:15:57.145311 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:57.145264 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" event={"ID":"85852870-ec89-4989-a187-94520d14393e","Type":"ContainerStarted","Data":"07b5f15354a0389acfe2899ae5c85063abe60a0a71cfa91666535076e85b5f19"} Jul 06 19:15:58.151616 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:58.151580 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" event={"ID":"85852870-ec89-4989-a187-94520d14393e","Type":"ContainerStarted","Data":"6754147c424752dad4eed6d8950a9edde5ca67666024b63861fba4c20442a440"} Jul 06 19:15:58.187681 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:58.187652 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:58.223606 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:15:58.223551 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" podStartSLOduration=1.202724414 podStartE2EDuration="2.223535608s" totalImagesPullingTime="1.020811194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:15:58.223431638 +0000 UTC m=+998.584595429" watchObservedRunningTime="2026-07-06 19:15:58.223535608 +0000 UTC m=+998.584699400" Jul 06 19:16:00.188507 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:00.188471 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:02.706378 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:02.706341 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:05.788135 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:05.788099 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:06.426856 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:06.426808 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:16:06.427055 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:06.426887 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:16:06.439707 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:06.439676 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:16:07.206541 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:07.206512 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wnjmx" Jul 06 19:16:07.887330 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:07.887282 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:08.329803 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:08.329767 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:10.047160 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:10.047120 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:12.691735 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:12.691700 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:13.678143 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:13.678109 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:21.112884 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:21.112851 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:21.489588 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:21.489559 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:24.451653 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:24.451612 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:29.240597 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:29.240553 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:48.400334 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:48.400294 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:52.169774 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:16:52.169720 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:29.913910 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:17:29.913879 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:43.339313 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:17:43.339273 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:44.213127 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:17:44.213091 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:46.247132 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:17:46.247098 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:46.929549 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:17:46.929506 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:47.915659 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:17:47.915624 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:48.405502 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:17:48.405457 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:51.943914 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:17:51.943878 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:18:18.735195 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:18:18.735156 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:05.399907 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:19:05.399807 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:07.225979 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:19:07.225946 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:34.740089 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:19:34.740048 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:34.924686 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:19:34.924651 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:35.102419 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:19:35.102326 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:43.255950 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:19:43.255911 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:43.429239 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:19:43.429196 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:43.675216 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:19:43.675101 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:51.181915 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:19:51.181879 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:51.396875 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:19:51.396842 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:31.847900 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:21:31.847867 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:33.259053 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:21:33.259011 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:38.480343 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:21:38.480305 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:40.118410 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:21:40.118356 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:23.114294 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:22:23.114253 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:04.552799 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:25:04.552717 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:14.100535 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:25:14.100501 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:14.197730 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:25:14.197696 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:51.024219 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:27:51.024140 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:54.610048 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:28:54.610014 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:54.819620 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:28:54.819589 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:07.049588 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:30:07.049553 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:07.915707 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:30:07.915672 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:19.095991 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:19.095958 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:26.722990 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:26.722957 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:35.383522 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:35.383486 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:40.686660 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:40.686622 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:43.581929 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:43.581885 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:43.697379 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:43.697350 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:44.032671 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:44.032639 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:50.905486 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:50.905430 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:51.083752 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:51.083694 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:51.560344 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:51.560234 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:51.853953 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:51.853873 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:52.340450 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.340410 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:52.539328 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.539297 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v"] Jul 06 19:31:52.542844 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.542823 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" Jul 06 19:31:52.545085 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.545054 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:52.547138 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.547120 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-a711c31a-serving-cert\"" Jul 06 19:31:52.547251 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.547182 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-zxgdj\"" Jul 06 19:31:52.575072 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.575045 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v"] Jul 06 19:31:52.621571 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.621482 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xvww\" (UniqueName: \"kubernetes.io/projected/410d49e3-75ce-4360-bb92-8238abd47e5e-kube-api-access-5xvww\") pod \"maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v\" (UID: \"410d49e3-75ce-4360-bb92-8238abd47e5e\") " pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" Jul 06 19:31:52.621722 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.621612 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/410d49e3-75ce-4360-bb92-8238abd47e5e-openshift-service-ca\") pod \"maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v\" (UID: \"410d49e3-75ce-4360-bb92-8238abd47e5e\") " pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" Jul 06 19:31:52.621722 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.621670 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/410d49e3-75ce-4360-bb92-8238abd47e5e-maas-api-tls\") pod \"maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v\" (UID: \"410d49e3-75ce-4360-bb92-8238abd47e5e\") " pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" Jul 06 19:31:52.695834 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.695798 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:52.722197 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.722164 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/410d49e3-75ce-4360-bb92-8238abd47e5e-openshift-service-ca\") pod \"maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v\" (UID: \"410d49e3-75ce-4360-bb92-8238abd47e5e\") " pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" Jul 06 19:31:52.722358 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.722207 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/410d49e3-75ce-4360-bb92-8238abd47e5e-maas-api-tls\") pod \"maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v\" (UID: \"410d49e3-75ce-4360-bb92-8238abd47e5e\") " pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" Jul 06 19:31:52.722358 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.722248 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xvww\" (UniqueName: \"kubernetes.io/projected/410d49e3-75ce-4360-bb92-8238abd47e5e-kube-api-access-5xvww\") pod \"maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v\" (UID: \"410d49e3-75ce-4360-bb92-8238abd47e5e\") " pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" Jul 06 19:31:52.722815 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.722788 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/410d49e3-75ce-4360-bb92-8238abd47e5e-openshift-service-ca\") pod \"maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v\" (UID: \"410d49e3-75ce-4360-bb92-8238abd47e5e\") " pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" Jul 06 19:31:52.724663 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.724646 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/410d49e3-75ce-4360-bb92-8238abd47e5e-maas-api-tls\") pod \"maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v\" (UID: \"410d49e3-75ce-4360-bb92-8238abd47e5e\") " pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" Jul 06 19:31:52.736818 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.736790 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xvww\" (UniqueName: \"kubernetes.io/projected/410d49e3-75ce-4360-bb92-8238abd47e5e-kube-api-access-5xvww\") pod \"maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v\" (UID: \"410d49e3-75ce-4360-bb92-8238abd47e5e\") " pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" Jul 06 19:31:52.853304 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.853271 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" Jul 06 19:31:52.926548 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:52.926521 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:53.009737 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:31:53.009709 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod410d49e3_75ce_4360_bb92_8238abd47e5e.slice/crio-4d8672c6adfa7ed26f127de00899423cc20bb6b54ebe0c04c7e23b90b112d392 WatchSource:0}: Error finding container 4d8672c6adfa7ed26f127de00899423cc20bb6b54ebe0c04c7e23b90b112d392: Status 404 returned error can't find the container with id 4d8672c6adfa7ed26f127de00899423cc20bb6b54ebe0c04c7e23b90b112d392 Jul 06 19:31:53.011277 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:53.011258 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:31:53.014837 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:53.014812 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v"] Jul 06 19:31:53.725562 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:53.725529 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:53.849792 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:53.849752 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" event={"ID":"410d49e3-75ce-4360-bb92-8238abd47e5e","Type":"ContainerStarted","Data":"93379e7c72d10a0794a3dd5ad9a2fe898a04e0030fa9b8a7ebcaf44df24e207a"} Jul 06 19:31:53.849979 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:53.849859 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" event={"ID":"410d49e3-75ce-4360-bb92-8238abd47e5e","Type":"ContainerStarted","Data":"4d8672c6adfa7ed26f127de00899423cc20bb6b54ebe0c04c7e23b90b112d392"} Jul 06 19:31:53.849979 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:53.849893 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" Jul 06 19:31:53.984453 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:53.984335 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" podStartSLOduration=1.378286629 podStartE2EDuration="1.984320671s" totalImagesPullingTime="606.034042ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:31:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:31:53.98162141 +0000 UTC m=+1954.342785202" watchObservedRunningTime="2026-07-06 19:31:53.984320671 +0000 UTC m=+1954.345484462" Jul 06 19:31:54.191124 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:54.191096 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:55.670619 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:55.670588 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:57.427050 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:57.427018 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:57.706684 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:57.706598 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:57.833416 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:57.833381 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:58.043151 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.043112 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v"] Jul 06 19:31:58.043405 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.043380 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" podUID="410d49e3-75ce-4360-bb92-8238abd47e5e" containerName="maas-api" containerID="cri-o://93379e7c72d10a0794a3dd5ad9a2fe898a04e0030fa9b8a7ebcaf44df24e207a" gracePeriod=30 Jul 06 19:31:58.048619 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.048596 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" Jul 06 19:31:58.170758 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.170687 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:58.302162 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.302103 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" Jul 06 19:31:58.371716 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.371691 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xvww\" (UniqueName: \"kubernetes.io/projected/410d49e3-75ce-4360-bb92-8238abd47e5e-kube-api-access-5xvww\") pod \"410d49e3-75ce-4360-bb92-8238abd47e5e\" (UID: \"410d49e3-75ce-4360-bb92-8238abd47e5e\") " Jul 06 19:31:58.371879 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.371732 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/410d49e3-75ce-4360-bb92-8238abd47e5e-openshift-service-ca\") pod \"410d49e3-75ce-4360-bb92-8238abd47e5e\" (UID: \"410d49e3-75ce-4360-bb92-8238abd47e5e\") " Jul 06 19:31:58.371945 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.371926 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/410d49e3-75ce-4360-bb92-8238abd47e5e-maas-api-tls\") pod \"410d49e3-75ce-4360-bb92-8238abd47e5e\" (UID: \"410d49e3-75ce-4360-bb92-8238abd47e5e\") " Jul 06 19:31:58.372110 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.372088 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/410d49e3-75ce-4360-bb92-8238abd47e5e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "410d49e3-75ce-4360-bb92-8238abd47e5e" (UID: "410d49e3-75ce-4360-bb92-8238abd47e5e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:31:58.372283 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.372270 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/410d49e3-75ce-4360-bb92-8238abd47e5e-openshift-service-ca\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:31:58.373774 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.373751 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/410d49e3-75ce-4360-bb92-8238abd47e5e-kube-api-access-5xvww" (OuterVolumeSpecName: "kube-api-access-5xvww") pod "410d49e3-75ce-4360-bb92-8238abd47e5e" (UID: "410d49e3-75ce-4360-bb92-8238abd47e5e"). InnerVolumeSpecName "kube-api-access-5xvww". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:31:58.373934 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.373918 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/410d49e3-75ce-4360-bb92-8238abd47e5e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "410d49e3-75ce-4360-bb92-8238abd47e5e" (UID: "410d49e3-75ce-4360-bb92-8238abd47e5e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:31:58.473617 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.473583 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/410d49e3-75ce-4360-bb92-8238abd47e5e-maas-api-tls\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:31:58.473617 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.473610 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xvww\" (UniqueName: \"kubernetes.io/projected/410d49e3-75ce-4360-bb92-8238abd47e5e-kube-api-access-5xvww\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:31:58.875683 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.875644 2557 generic.go:356] "Generic (PLEG): container finished" podID="410d49e3-75ce-4360-bb92-8238abd47e5e" containerID="93379e7c72d10a0794a3dd5ad9a2fe898a04e0030fa9b8a7ebcaf44df24e207a" exitCode=0 Jul 06 19:31:58.875857 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.875686 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" Jul 06 19:31:58.875857 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.875725 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" event={"ID":"410d49e3-75ce-4360-bb92-8238abd47e5e","Type":"ContainerDied","Data":"93379e7c72d10a0794a3dd5ad9a2fe898a04e0030fa9b8a7ebcaf44df24e207a"} Jul 06 19:31:58.875857 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.875773 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v" event={"ID":"410d49e3-75ce-4360-bb92-8238abd47e5e","Type":"ContainerDied","Data":"4d8672c6adfa7ed26f127de00899423cc20bb6b54ebe0c04c7e23b90b112d392"} Jul 06 19:31:58.875857 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.875794 2557 scope.go:117] "RemoveContainer" containerID="93379e7c72d10a0794a3dd5ad9a2fe898a04e0030fa9b8a7ebcaf44df24e207a" Jul 06 19:31:58.885534 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.885514 2557 scope.go:117] "RemoveContainer" containerID="93379e7c72d10a0794a3dd5ad9a2fe898a04e0030fa9b8a7ebcaf44df24e207a" Jul 06 19:31:58.885811 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:31:58.885786 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93379e7c72d10a0794a3dd5ad9a2fe898a04e0030fa9b8a7ebcaf44df24e207a\": container with ID starting with 93379e7c72d10a0794a3dd5ad9a2fe898a04e0030fa9b8a7ebcaf44df24e207a not found: ID does not exist" containerID="93379e7c72d10a0794a3dd5ad9a2fe898a04e0030fa9b8a7ebcaf44df24e207a" Jul 06 19:31:58.885892 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.885818 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93379e7c72d10a0794a3dd5ad9a2fe898a04e0030fa9b8a7ebcaf44df24e207a"} err="failed to get container status \"93379e7c72d10a0794a3dd5ad9a2fe898a04e0030fa9b8a7ebcaf44df24e207a\": rpc error: code = NotFound desc = could not find container \"93379e7c72d10a0794a3dd5ad9a2fe898a04e0030fa9b8a7ebcaf44df24e207a\": container with ID starting with 93379e7c72d10a0794a3dd5ad9a2fe898a04e0030fa9b8a7ebcaf44df24e207a not found: ID does not exist" Jul 06 19:31:58.910327 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.910297 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v"] Jul 06 19:31:58.920629 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:58.920602 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-a711c31a-5c595cf7f-qg99v"] Jul 06 19:31:59.008714 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:31:59.008681 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:00.197051 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:00.197018 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="410d49e3-75ce-4360-bb92-8238abd47e5e" path="/var/lib/kubelet/pods/410d49e3-75ce-4360-bb92-8238abd47e5e/volumes" Jul 06 19:32:04.666211 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:04.666181 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:04.780260 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:04.780204 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:04.951627 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:04.951545 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:05.074066 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:05.074036 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:06.166921 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.166889 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:06.338466 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.338422 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf"] Jul 06 19:32:06.338851 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.338839 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="410d49e3-75ce-4360-bb92-8238abd47e5e" containerName="maas-api" Jul 06 19:32:06.338898 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.338853 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="410d49e3-75ce-4360-bb92-8238abd47e5e" containerName="maas-api" Jul 06 19:32:06.338941 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.338924 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="410d49e3-75ce-4360-bb92-8238abd47e5e" containerName="maas-api" Jul 06 19:32:06.343418 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.343398 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" Jul 06 19:32:06.347140 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.347121 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-zxgdj\"" Jul 06 19:32:06.347140 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.347130 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-d972dd58-serving-cert\"" Jul 06 19:32:06.354271 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.354253 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:06.361090 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.361066 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf"] Jul 06 19:32:06.447197 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.447105 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3b8362-03dc-4441-b8d9-085ad467e6b2-openshift-service-ca\") pod \"maas-api-e2e-ait-d972dd58-779876664d-dmzvf\" (UID: \"3b3b8362-03dc-4441-b8d9-085ad467e6b2\") " pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" Jul 06 19:32:06.447197 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.447157 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27q4h\" (UniqueName: \"kubernetes.io/projected/3b3b8362-03dc-4441-b8d9-085ad467e6b2-kube-api-access-27q4h\") pod \"maas-api-e2e-ait-d972dd58-779876664d-dmzvf\" (UID: \"3b3b8362-03dc-4441-b8d9-085ad467e6b2\") " pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" Jul 06 19:32:06.447374 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.447245 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b3b8362-03dc-4441-b8d9-085ad467e6b2-maas-api-tls\") pod \"maas-api-e2e-ait-d972dd58-779876664d-dmzvf\" (UID: \"3b3b8362-03dc-4441-b8d9-085ad467e6b2\") " pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" Jul 06 19:32:06.456705 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.456675 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:06.547928 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.547895 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3b8362-03dc-4441-b8d9-085ad467e6b2-openshift-service-ca\") pod \"maas-api-e2e-ait-d972dd58-779876664d-dmzvf\" (UID: \"3b3b8362-03dc-4441-b8d9-085ad467e6b2\") " pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" Jul 06 19:32:06.548126 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.547943 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27q4h\" (UniqueName: \"kubernetes.io/projected/3b3b8362-03dc-4441-b8d9-085ad467e6b2-kube-api-access-27q4h\") pod \"maas-api-e2e-ait-d972dd58-779876664d-dmzvf\" (UID: \"3b3b8362-03dc-4441-b8d9-085ad467e6b2\") " pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" Jul 06 19:32:06.548126 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.547984 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b3b8362-03dc-4441-b8d9-085ad467e6b2-maas-api-tls\") pod \"maas-api-e2e-ait-d972dd58-779876664d-dmzvf\" (UID: \"3b3b8362-03dc-4441-b8d9-085ad467e6b2\") " pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" Jul 06 19:32:06.548646 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.548620 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3b8362-03dc-4441-b8d9-085ad467e6b2-openshift-service-ca\") pod \"maas-api-e2e-ait-d972dd58-779876664d-dmzvf\" (UID: \"3b3b8362-03dc-4441-b8d9-085ad467e6b2\") " pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" Jul 06 19:32:06.550481 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.550464 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b3b8362-03dc-4441-b8d9-085ad467e6b2-maas-api-tls\") pod \"maas-api-e2e-ait-d972dd58-779876664d-dmzvf\" (UID: \"3b3b8362-03dc-4441-b8d9-085ad467e6b2\") " pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" Jul 06 19:32:06.562162 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.562141 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27q4h\" (UniqueName: \"kubernetes.io/projected/3b3b8362-03dc-4441-b8d9-085ad467e6b2-kube-api-access-27q4h\") pod \"maas-api-e2e-ait-d972dd58-779876664d-dmzvf\" (UID: \"3b3b8362-03dc-4441-b8d9-085ad467e6b2\") " pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" Jul 06 19:32:06.580783 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.580760 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:06.654080 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.654048 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" Jul 06 19:32:06.804081 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.804048 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf"] Jul 06 19:32:06.806575 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:32:06.806546 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b3b8362_03dc_4441_b8d9_085ad467e6b2.slice/crio-616b111495c9568d7c346148733cf48806cf3cb62ff6e293d5d7bfae16f5e8fc WatchSource:0}: Error finding container 616b111495c9568d7c346148733cf48806cf3cb62ff6e293d5d7bfae16f5e8fc: Status 404 returned error can't find the container with id 616b111495c9568d7c346148733cf48806cf3cb62ff6e293d5d7bfae16f5e8fc Jul 06 19:32:06.918107 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:06.918069 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" event={"ID":"3b3b8362-03dc-4441-b8d9-085ad467e6b2","Type":"ContainerStarted","Data":"616b111495c9568d7c346148733cf48806cf3cb62ff6e293d5d7bfae16f5e8fc"} Jul 06 19:32:07.048284 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:07.048251 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:07.150377 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:07.150345 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:07.924295 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:07.924255 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" event={"ID":"3b3b8362-03dc-4441-b8d9-085ad467e6b2","Type":"ContainerStarted","Data":"ff34f3a7fe348c8ce169631719149405ff3bd66e55bfd3da45d65f705b24a2fb"} Jul 06 19:32:07.924781 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:07.924328 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" Jul 06 19:32:07.948748 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:07.948697 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" podStartSLOduration=1.530102003 podStartE2EDuration="1.948681579s" totalImagesPullingTime="418.579576ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:32:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:32:07.947567051 +0000 UTC m=+1968.308730844" watchObservedRunningTime="2026-07-06 19:32:07.948681579 +0000 UTC m=+1968.309845371" Jul 06 19:32:08.756859 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:08.756815 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:13.934953 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:13.934873 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" Jul 06 19:32:27.693860 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:27.693823 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:28.857854 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:28.857816 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf"] Jul 06 19:32:28.858286 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:28.858016 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" podUID="3b3b8362-03dc-4441-b8d9-085ad467e6b2" containerName="maas-api" containerID="cri-o://ff34f3a7fe348c8ce169631719149405ff3bd66e55bfd3da45d65f705b24a2fb" gracePeriod=30 Jul 06 19:32:28.929381 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:28.929336 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" podUID="3b3b8362-03dc-4441-b8d9-085ad467e6b2" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.36:8443/health\": dial tcp 10.134.0.36:8443: connect: connection refused" Jul 06 19:32:28.957067 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:28.957037 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:29.031327 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:29.031291 2557 generic.go:356] "Generic (PLEG): container finished" podID="3b3b8362-03dc-4441-b8d9-085ad467e6b2" containerID="ff34f3a7fe348c8ce169631719149405ff3bd66e55bfd3da45d65f705b24a2fb" exitCode=0 Jul 06 19:32:29.031487 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:29.031356 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" event={"ID":"3b3b8362-03dc-4441-b8d9-085ad467e6b2","Type":"ContainerDied","Data":"ff34f3a7fe348c8ce169631719149405ff3bd66e55bfd3da45d65f705b24a2fb"} Jul 06 19:32:29.120659 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:29.120600 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" Jul 06 19:32:29.266055 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:29.265997 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3b8362-03dc-4441-b8d9-085ad467e6b2-openshift-service-ca\") pod \"3b3b8362-03dc-4441-b8d9-085ad467e6b2\" (UID: \"3b3b8362-03dc-4441-b8d9-085ad467e6b2\") " Jul 06 19:32:29.266275 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:29.266175 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27q4h\" (UniqueName: \"kubernetes.io/projected/3b3b8362-03dc-4441-b8d9-085ad467e6b2-kube-api-access-27q4h\") pod \"3b3b8362-03dc-4441-b8d9-085ad467e6b2\" (UID: \"3b3b8362-03dc-4441-b8d9-085ad467e6b2\") " Jul 06 19:32:29.266275 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:29.266219 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b3b8362-03dc-4441-b8d9-085ad467e6b2-maas-api-tls\") pod \"3b3b8362-03dc-4441-b8d9-085ad467e6b2\" (UID: \"3b3b8362-03dc-4441-b8d9-085ad467e6b2\") " Jul 06 19:32:29.266389 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:29.266342 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b3b8362-03dc-4441-b8d9-085ad467e6b2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b3b8362-03dc-4441-b8d9-085ad467e6b2" (UID: "3b3b8362-03dc-4441-b8d9-085ad467e6b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:32:29.266553 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:29.266535 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3b8362-03dc-4441-b8d9-085ad467e6b2-openshift-service-ca\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:32:29.268298 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:29.268266 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b3b8362-03dc-4441-b8d9-085ad467e6b2-kube-api-access-27q4h" (OuterVolumeSpecName: "kube-api-access-27q4h") pod "3b3b8362-03dc-4441-b8d9-085ad467e6b2" (UID: "3b3b8362-03dc-4441-b8d9-085ad467e6b2"). InnerVolumeSpecName "kube-api-access-27q4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:32:29.268415 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:29.268376 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b3b8362-03dc-4441-b8d9-085ad467e6b2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b3b8362-03dc-4441-b8d9-085ad467e6b2" (UID: "3b3b8362-03dc-4441-b8d9-085ad467e6b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:32:29.367385 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:29.367346 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b3b8362-03dc-4441-b8d9-085ad467e6b2-maas-api-tls\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:32:29.367385 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:29.367382 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27q4h\" (UniqueName: \"kubernetes.io/projected/3b3b8362-03dc-4441-b8d9-085ad467e6b2-kube-api-access-27q4h\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:32:30.037647 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:30.037621 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" Jul 06 19:32:30.038122 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:30.037621 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf" event={"ID":"3b3b8362-03dc-4441-b8d9-085ad467e6b2","Type":"ContainerDied","Data":"616b111495c9568d7c346148733cf48806cf3cb62ff6e293d5d7bfae16f5e8fc"} Jul 06 19:32:30.038122 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:30.037761 2557 scope.go:117] "RemoveContainer" containerID="ff34f3a7fe348c8ce169631719149405ff3bd66e55bfd3da45d65f705b24a2fb" Jul 06 19:32:30.073913 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:30.073883 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf"] Jul 06 19:32:30.075468 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:30.075446 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-d972dd58-779876664d-dmzvf"] Jul 06 19:32:30.196947 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:30.196916 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b3b8362-03dc-4441-b8d9-085ad467e6b2" path="/var/lib/kubelet/pods/3b3b8362-03dc-4441-b8d9-085ad467e6b2/volumes" Jul 06 19:32:33.950976 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:33.950934 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:34.090893 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:34.090859 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:40.813296 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:40.813243 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:40.894699 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:40.894664 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:42.173790 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.173757 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:42.297405 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.297366 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj"] Jul 06 19:32:42.297857 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.297839 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b3b8362-03dc-4441-b8d9-085ad467e6b2" containerName="maas-api" Jul 06 19:32:42.297946 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.297861 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b3b8362-03dc-4441-b8d9-085ad467e6b2" containerName="maas-api" Jul 06 19:32:42.298006 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.297986 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b3b8362-03dc-4441-b8d9-085ad467e6b2" containerName="maas-api" Jul 06 19:32:42.302289 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.302269 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" Jul 06 19:32:42.308097 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.308069 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-ec6a8779-serving-cert\"" Jul 06 19:32:42.308741 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.308524 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-zxgdj\"" Jul 06 19:32:42.352702 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.352670 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj"] Jul 06 19:32:42.384329 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.384290 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjlsm\" (UniqueName: \"kubernetes.io/projected/b6b4ca35-c467-4802-8b8b-17f7824c54cf-kube-api-access-hjlsm\") pod \"maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj\" (UID: \"b6b4ca35-c467-4802-8b8b-17f7824c54cf\") " pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" Jul 06 19:32:42.384522 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.384349 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6b4ca35-c467-4802-8b8b-17f7824c54cf-openshift-service-ca\") pod \"maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj\" (UID: \"b6b4ca35-c467-4802-8b8b-17f7824c54cf\") " pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" Jul 06 19:32:42.384522 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.384463 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6b4ca35-c467-4802-8b8b-17f7824c54cf-maas-api-tls\") pod \"maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj\" (UID: \"b6b4ca35-c467-4802-8b8b-17f7824c54cf\") " pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" Jul 06 19:32:42.485877 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.485778 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6b4ca35-c467-4802-8b8b-17f7824c54cf-maas-api-tls\") pod \"maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj\" (UID: \"b6b4ca35-c467-4802-8b8b-17f7824c54cf\") " pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" Jul 06 19:32:42.485877 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.485828 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjlsm\" (UniqueName: \"kubernetes.io/projected/b6b4ca35-c467-4802-8b8b-17f7824c54cf-kube-api-access-hjlsm\") pod \"maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj\" (UID: \"b6b4ca35-c467-4802-8b8b-17f7824c54cf\") " pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" Jul 06 19:32:42.486124 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.485883 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6b4ca35-c467-4802-8b8b-17f7824c54cf-openshift-service-ca\") pod \"maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj\" (UID: \"b6b4ca35-c467-4802-8b8b-17f7824c54cf\") " pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" Jul 06 19:32:42.486124 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:32:42.485934 2557 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-ec6a8779-serving-cert: secret "maas-api-e2e-derive-ec6a8779-serving-cert" not found Jul 06 19:32:42.486124 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:32:42.486000 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6b4ca35-c467-4802-8b8b-17f7824c54cf-maas-api-tls podName:b6b4ca35-c467-4802-8b8b-17f7824c54cf nodeName:}" failed. No retries permitted until 2026-07-06 19:32:42.985982471 +0000 UTC m=+2003.347146241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b6b4ca35-c467-4802-8b8b-17f7824c54cf-maas-api-tls") pod "maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" (UID: "b6b4ca35-c467-4802-8b8b-17f7824c54cf") : secret "maas-api-e2e-derive-ec6a8779-serving-cert" not found Jul 06 19:32:42.486627 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.486605 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6b4ca35-c467-4802-8b8b-17f7824c54cf-openshift-service-ca\") pod \"maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj\" (UID: \"b6b4ca35-c467-4802-8b8b-17f7824c54cf\") " pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" Jul 06 19:32:42.502761 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.502741 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjlsm\" (UniqueName: \"kubernetes.io/projected/b6b4ca35-c467-4802-8b8b-17f7824c54cf-kube-api-access-hjlsm\") pod \"maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj\" (UID: \"b6b4ca35-c467-4802-8b8b-17f7824c54cf\") " pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" Jul 06 19:32:42.531729 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.531697 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:42.912209 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.912174 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:42.992064 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.992028 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6b4ca35-c467-4802-8b8b-17f7824c54cf-maas-api-tls\") pod \"maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj\" (UID: \"b6b4ca35-c467-4802-8b8b-17f7824c54cf\") " pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" Jul 06 19:32:42.994541 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:42.994514 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6b4ca35-c467-4802-8b8b-17f7824c54cf-maas-api-tls\") pod \"maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj\" (UID: \"b6b4ca35-c467-4802-8b8b-17f7824c54cf\") " pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" Jul 06 19:32:43.028471 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:43.028425 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:43.146163 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:43.146108 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:43.212008 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:43.211908 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" Jul 06 19:32:43.271603 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:43.271558 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:43.363612 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:43.363581 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj"] Jul 06 19:32:43.366324 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:32:43.366303 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6b4ca35_c467_4802_8b8b_17f7824c54cf.slice/crio-a264b7b8bdaaaa24d560f954273ac102d690446be000265a2b067918d5d92b4a WatchSource:0}: Error finding container a264b7b8bdaaaa24d560f954273ac102d690446be000265a2b067918d5d92b4a: Status 404 returned error can't find the container with id a264b7b8bdaaaa24d560f954273ac102d690446be000265a2b067918d5d92b4a Jul 06 19:32:43.486322 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:43.486293 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:43.592660 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:43.592628 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:43.983016 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:43.982982 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:44.109980 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:44.109885 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" event={"ID":"b6b4ca35-c467-4802-8b8b-17f7824c54cf","Type":"ContainerStarted","Data":"7b5d94153d85ddc8a351b5e1e980e159429d722ac37dc64472bd12a1146165a8"} Jul 06 19:32:44.109980 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:44.109944 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" event={"ID":"b6b4ca35-c467-4802-8b8b-17f7824c54cf","Type":"ContainerStarted","Data":"a264b7b8bdaaaa24d560f954273ac102d690446be000265a2b067918d5d92b4a"} Jul 06 19:32:44.110161 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:44.109993 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" Jul 06 19:32:46.878934 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:46.878902 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:47.091759 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:47.091726 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:47.320293 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:47.320257 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:47.556842 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:47.556747 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:47.565613 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:47.565562 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" podStartSLOduration=5.170688305 podStartE2EDuration="5.565545842s" totalImagesPullingTime="394.857537ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:32:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:32:44.150673508 +0000 UTC m=+2004.511837300" watchObservedRunningTime="2026-07-06 19:32:47.565545842 +0000 UTC m=+2007.926709634" Jul 06 19:32:47.566200 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:47.566185 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj"] Jul 06 19:32:47.566411 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:47.566391 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" podUID="b6b4ca35-c467-4802-8b8b-17f7824c54cf" containerName="maas-api" containerID="cri-o://7b5d94153d85ddc8a351b5e1e980e159429d722ac37dc64472bd12a1146165a8" gracePeriod=30 Jul 06 19:32:47.721217 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:47.721180 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:47.823201 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:47.823177 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" Jul 06 19:32:47.942715 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:47.942682 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjlsm\" (UniqueName: \"kubernetes.io/projected/b6b4ca35-c467-4802-8b8b-17f7824c54cf-kube-api-access-hjlsm\") pod \"b6b4ca35-c467-4802-8b8b-17f7824c54cf\" (UID: \"b6b4ca35-c467-4802-8b8b-17f7824c54cf\") " Jul 06 19:32:47.943117 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:47.942782 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6b4ca35-c467-4802-8b8b-17f7824c54cf-openshift-service-ca\") pod \"b6b4ca35-c467-4802-8b8b-17f7824c54cf\" (UID: \"b6b4ca35-c467-4802-8b8b-17f7824c54cf\") " Jul 06 19:32:47.943117 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:47.942821 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6b4ca35-c467-4802-8b8b-17f7824c54cf-maas-api-tls\") pod \"b6b4ca35-c467-4802-8b8b-17f7824c54cf\" (UID: \"b6b4ca35-c467-4802-8b8b-17f7824c54cf\") " Jul 06 19:32:47.943248 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:47.943167 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6b4ca35-c467-4802-8b8b-17f7824c54cf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b6b4ca35-c467-4802-8b8b-17f7824c54cf" (UID: "b6b4ca35-c467-4802-8b8b-17f7824c54cf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:32:47.944916 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:47.944891 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6b4ca35-c467-4802-8b8b-17f7824c54cf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b6b4ca35-c467-4802-8b8b-17f7824c54cf" (UID: "b6b4ca35-c467-4802-8b8b-17f7824c54cf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:32:47.945126 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:47.945104 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6b4ca35-c467-4802-8b8b-17f7824c54cf-kube-api-access-hjlsm" (OuterVolumeSpecName: "kube-api-access-hjlsm") pod "b6b4ca35-c467-4802-8b8b-17f7824c54cf" (UID: "b6b4ca35-c467-4802-8b8b-17f7824c54cf"). InnerVolumeSpecName "kube-api-access-hjlsm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:32:48.043988 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:48.043948 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjlsm\" (UniqueName: \"kubernetes.io/projected/b6b4ca35-c467-4802-8b8b-17f7824c54cf-kube-api-access-hjlsm\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:32:48.043988 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:48.043980 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6b4ca35-c467-4802-8b8b-17f7824c54cf-openshift-service-ca\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:32:48.043988 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:48.043990 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6b4ca35-c467-4802-8b8b-17f7824c54cf-maas-api-tls\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:32:48.132550 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:48.132458 2557 generic.go:356] "Generic (PLEG): container finished" podID="b6b4ca35-c467-4802-8b8b-17f7824c54cf" containerID="7b5d94153d85ddc8a351b5e1e980e159429d722ac37dc64472bd12a1146165a8" exitCode=0 Jul 06 19:32:48.132550 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:48.132504 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" Jul 06 19:32:48.132550 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:48.132514 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" event={"ID":"b6b4ca35-c467-4802-8b8b-17f7824c54cf","Type":"ContainerDied","Data":"7b5d94153d85ddc8a351b5e1e980e159429d722ac37dc64472bd12a1146165a8"} Jul 06 19:32:48.132794 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:48.132557 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj" event={"ID":"b6b4ca35-c467-4802-8b8b-17f7824c54cf","Type":"ContainerDied","Data":"a264b7b8bdaaaa24d560f954273ac102d690446be000265a2b067918d5d92b4a"} Jul 06 19:32:48.132794 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:48.132578 2557 scope.go:117] "RemoveContainer" containerID="7b5d94153d85ddc8a351b5e1e980e159429d722ac37dc64472bd12a1146165a8" Jul 06 19:32:48.142061 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:48.142041 2557 scope.go:117] "RemoveContainer" containerID="7b5d94153d85ddc8a351b5e1e980e159429d722ac37dc64472bd12a1146165a8" Jul 06 19:32:48.142328 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:32:48.142308 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b5d94153d85ddc8a351b5e1e980e159429d722ac37dc64472bd12a1146165a8\": container with ID starting with 7b5d94153d85ddc8a351b5e1e980e159429d722ac37dc64472bd12a1146165a8 not found: ID does not exist" containerID="7b5d94153d85ddc8a351b5e1e980e159429d722ac37dc64472bd12a1146165a8" Jul 06 19:32:48.142395 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:48.142343 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b5d94153d85ddc8a351b5e1e980e159429d722ac37dc64472bd12a1146165a8"} err="failed to get container status \"7b5d94153d85ddc8a351b5e1e980e159429d722ac37dc64472bd12a1146165a8\": rpc error: code = NotFound desc = could not find container \"7b5d94153d85ddc8a351b5e1e980e159429d722ac37dc64472bd12a1146165a8\": container with ID starting with 7b5d94153d85ddc8a351b5e1e980e159429d722ac37dc64472bd12a1146165a8 not found: ID does not exist" Jul 06 19:32:48.162854 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:48.162826 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj"] Jul 06 19:32:48.167959 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:48.167936 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-ec6a8779-5b9d949976-xxzrj"] Jul 06 19:32:48.197485 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:48.197461 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6b4ca35-c467-4802-8b8b-17f7824c54cf" path="/var/lib/kubelet/pods/b6b4ca35-c467-4802-8b8b-17f7824c54cf/volumes" Jul 06 19:32:50.318850 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:50.318800 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:54.860070 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:54.860031 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:56.464503 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:56.464471 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:57.419799 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:32:57.419767 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:07.502328 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:33:07.502286 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:07.867059 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:33:07.866974 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:08.000712 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:33:08.000680 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:16.365140 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:33:16.365091 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:33.180359 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:33:33.180326 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:39.587627 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:33:39.587593 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:39.712187 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:33:39.712156 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:39.825353 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:33:39.825311 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:39.930672 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:33:39.930586 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:40.037650 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:33:40.037619 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:55.772206 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:33:55.772114 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:56.659750 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:33:56.659722 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:02.545055 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:02.545025 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:02.668545 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:02.668516 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:02.866906 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:02.866824 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:13.403628 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:13.403596 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:14.262116 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:14.262081 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:21.774411 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:21.774378 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:21.962298 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:21.962264 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:22.070792 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:22.070706 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:23.165954 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:23.165917 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:26.120663 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:26.120630 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:27.775903 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:27.775870 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:30.824043 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:30.824009 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:30.958785 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:30.958747 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:34.878400 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:34.878364 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:35.002732 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:35.002698 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:35.605903 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:35.605869 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:42.257280 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:42.257251 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:42.350701 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:42.350642 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:42.495571 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:42.495543 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:42.880660 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:42.880625 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:43.389878 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:43.389844 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:46.128940 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.128897 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:46.266874 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.266845 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:46.279178 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.279144 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn"] Jul 06 19:34:46.279558 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.279546 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6b4ca35-c467-4802-8b8b-17f7824c54cf" containerName="maas-api" Jul 06 19:34:46.279619 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.279560 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6b4ca35-c467-4802-8b8b-17f7824c54cf" containerName="maas-api" Jul 06 19:34:46.279655 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.279625 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6b4ca35-c467-4802-8b8b-17f7824c54cf" containerName="maas-api" Jul 06 19:34:46.282790 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.282775 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" Jul 06 19:34:46.286334 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.286317 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-95f72a38-serving-cert\"" Jul 06 19:34:46.286423 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.286370 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-zxgdj\"" Jul 06 19:34:46.305714 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.305682 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn"] Jul 06 19:34:46.358467 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.358406 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55ca23a-5320-4054-9e73-bf0523c517ba-openshift-service-ca\") pod \"maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn\" (UID: \"b55ca23a-5320-4054-9e73-bf0523c517ba\") " pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" Jul 06 19:34:46.358664 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.358524 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55ca23a-5320-4054-9e73-bf0523c517ba-maas-api-tls\") pod \"maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn\" (UID: \"b55ca23a-5320-4054-9e73-bf0523c517ba\") " pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" Jul 06 19:34:46.358664 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.358582 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thsvc\" (UniqueName: \"kubernetes.io/projected/b55ca23a-5320-4054-9e73-bf0523c517ba-kube-api-access-thsvc\") pod \"maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn\" (UID: \"b55ca23a-5320-4054-9e73-bf0523c517ba\") " pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" Jul 06 19:34:46.395857 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.395773 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:46.459816 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.459778 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55ca23a-5320-4054-9e73-bf0523c517ba-maas-api-tls\") pod \"maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn\" (UID: \"b55ca23a-5320-4054-9e73-bf0523c517ba\") " pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" Jul 06 19:34:46.460017 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.459829 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thsvc\" (UniqueName: \"kubernetes.io/projected/b55ca23a-5320-4054-9e73-bf0523c517ba-kube-api-access-thsvc\") pod \"maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn\" (UID: \"b55ca23a-5320-4054-9e73-bf0523c517ba\") " pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" Jul 06 19:34:46.460017 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.459894 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55ca23a-5320-4054-9e73-bf0523c517ba-openshift-service-ca\") pod \"maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn\" (UID: \"b55ca23a-5320-4054-9e73-bf0523c517ba\") " pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" Jul 06 19:34:46.460587 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.460568 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55ca23a-5320-4054-9e73-bf0523c517ba-openshift-service-ca\") pod \"maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn\" (UID: \"b55ca23a-5320-4054-9e73-bf0523c517ba\") " pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" Jul 06 19:34:46.462227 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.462208 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55ca23a-5320-4054-9e73-bf0523c517ba-maas-api-tls\") pod \"maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn\" (UID: \"b55ca23a-5320-4054-9e73-bf0523c517ba\") " pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" Jul 06 19:34:46.483292 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.483258 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thsvc\" (UniqueName: \"kubernetes.io/projected/b55ca23a-5320-4054-9e73-bf0523c517ba-kube-api-access-thsvc\") pod \"maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn\" (UID: \"b55ca23a-5320-4054-9e73-bf0523c517ba\") " pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" Jul 06 19:34:46.550711 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.550671 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:46.592859 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.592827 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" Jul 06 19:34:46.752797 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:34:46.752761 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb55ca23a_5320_4054_9e73_bf0523c517ba.slice/crio-70149879947b412b7f5ab5013c7105f66165c692be6a7fadf90d6d1bf0a2e017 WatchSource:0}: Error finding container 70149879947b412b7f5ab5013c7105f66165c692be6a7fadf90d6d1bf0a2e017: Status 404 returned error can't find the container with id 70149879947b412b7f5ab5013c7105f66165c692be6a7fadf90d6d1bf0a2e017 Jul 06 19:34:46.757132 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:46.757108 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn"] Jul 06 19:34:47.504303 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:47.504270 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:47.722108 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:47.722076 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" event={"ID":"b55ca23a-5320-4054-9e73-bf0523c517ba","Type":"ContainerStarted","Data":"5b74b77d9ce6f9d9a798b786ce5784fba1ef451e4fe0e1c964103d628172e5cb"} Jul 06 19:34:47.722312 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:47.722132 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" event={"ID":"b55ca23a-5320-4054-9e73-bf0523c517ba","Type":"ContainerStarted","Data":"70149879947b412b7f5ab5013c7105f66165c692be6a7fadf90d6d1bf0a2e017"} Jul 06 19:34:47.722312 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:47.722158 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" Jul 06 19:34:47.914679 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:47.914578 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:50.561935 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:50.561896 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:51.232593 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:51.232556 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:51.326958 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:51.326926 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:51.679288 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:51.679253 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:53.733883 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:53.733854 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" Jul 06 19:34:53.760765 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:53.760709 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" podStartSLOduration=7.308149979 podStartE2EDuration="7.760692635s" totalImagesPullingTime="452.542656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:47.795060231 +0000 UTC m=+2128.156224025" watchObservedRunningTime="2026-07-06 19:34:53.760692635 +0000 UTC m=+2134.121856426" Jul 06 19:34:57.826194 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:57.826157 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:57.902724 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:57.902685 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:58.261828 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:58.261794 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:58.600836 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:58.600732 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn"] Jul 06 19:34:58.601042 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:58.601007 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" podUID="b55ca23a-5320-4054-9e73-bf0523c517ba" containerName="maas-api" containerID="cri-o://5b74b77d9ce6f9d9a798b786ce5784fba1ef451e4fe0e1c964103d628172e5cb" gracePeriod=30 Jul 06 19:34:58.628192 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:58.628161 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:58.698660 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:58.698613 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:58.728559 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:58.728524 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" podUID="b55ca23a-5320-4054-9e73-bf0523c517ba" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.38:8443/health\": dial tcp 10.134.0.38:8443: connect: connection refused" Jul 06 19:34:58.779527 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:58.779491 2557 generic.go:356] "Generic (PLEG): container finished" podID="b55ca23a-5320-4054-9e73-bf0523c517ba" containerID="5b74b77d9ce6f9d9a798b786ce5784fba1ef451e4fe0e1c964103d628172e5cb" exitCode=0 Jul 06 19:34:58.779669 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:58.779565 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" event={"ID":"b55ca23a-5320-4054-9e73-bf0523c517ba","Type":"ContainerDied","Data":"5b74b77d9ce6f9d9a798b786ce5784fba1ef451e4fe0e1c964103d628172e5cb"} Jul 06 19:34:58.845021 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:58.844996 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" Jul 06 19:34:58.986723 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:58.986691 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55ca23a-5320-4054-9e73-bf0523c517ba-openshift-service-ca\") pod \"b55ca23a-5320-4054-9e73-bf0523c517ba\" (UID: \"b55ca23a-5320-4054-9e73-bf0523c517ba\") " Jul 06 19:34:58.986907 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:58.986757 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55ca23a-5320-4054-9e73-bf0523c517ba-maas-api-tls\") pod \"b55ca23a-5320-4054-9e73-bf0523c517ba\" (UID: \"b55ca23a-5320-4054-9e73-bf0523c517ba\") " Jul 06 19:34:58.986907 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:58.986782 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thsvc\" (UniqueName: \"kubernetes.io/projected/b55ca23a-5320-4054-9e73-bf0523c517ba-kube-api-access-thsvc\") pod \"b55ca23a-5320-4054-9e73-bf0523c517ba\" (UID: \"b55ca23a-5320-4054-9e73-bf0523c517ba\") " Jul 06 19:34:58.987177 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:58.987150 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b55ca23a-5320-4054-9e73-bf0523c517ba-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b55ca23a-5320-4054-9e73-bf0523c517ba" (UID: "b55ca23a-5320-4054-9e73-bf0523c517ba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:34:58.988863 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:58.988841 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b55ca23a-5320-4054-9e73-bf0523c517ba-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b55ca23a-5320-4054-9e73-bf0523c517ba" (UID: "b55ca23a-5320-4054-9e73-bf0523c517ba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:34:58.989033 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:58.989011 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b55ca23a-5320-4054-9e73-bf0523c517ba-kube-api-access-thsvc" (OuterVolumeSpecName: "kube-api-access-thsvc") pod "b55ca23a-5320-4054-9e73-bf0523c517ba" (UID: "b55ca23a-5320-4054-9e73-bf0523c517ba"). InnerVolumeSpecName "kube-api-access-thsvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:34:59.087933 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:59.087890 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55ca23a-5320-4054-9e73-bf0523c517ba-openshift-service-ca\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:34:59.087933 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:59.087928 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55ca23a-5320-4054-9e73-bf0523c517ba-maas-api-tls\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:34:59.087933 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:59.087938 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thsvc\" (UniqueName: \"kubernetes.io/projected/b55ca23a-5320-4054-9e73-bf0523c517ba-kube-api-access-thsvc\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:34:59.786429 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:59.786393 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" event={"ID":"b55ca23a-5320-4054-9e73-bf0523c517ba","Type":"ContainerDied","Data":"70149879947b412b7f5ab5013c7105f66165c692be6a7fadf90d6d1bf0a2e017"} Jul 06 19:34:59.786429 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:59.786424 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn" Jul 06 19:34:59.786690 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:59.786476 2557 scope.go:117] "RemoveContainer" containerID="5b74b77d9ce6f9d9a798b786ce5784fba1ef451e4fe0e1c964103d628172e5cb" Jul 06 19:34:59.816539 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:59.816505 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn"] Jul 06 19:34:59.819806 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:34:59.819780 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-95f72a38-8665cf49f9-pdvnn"] Jul 06 19:35:00.196924 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:00.196842 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b55ca23a-5320-4054-9e73-bf0523c517ba" path="/var/lib/kubelet/pods/b55ca23a-5320-4054-9e73-bf0523c517ba/volumes" Jul 06 19:35:01.675138 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:01.675105 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:05.467570 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:05.467538 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:05.545143 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:05.545109 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:05.871142 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:05.871106 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:06.028205 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:06.028173 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:06.154259 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:06.154178 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:06.243081 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:06.243051 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:06.343229 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:06.343188 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:12.857659 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:12.857572 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:12.934064 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:12.934033 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:13.030549 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:13.030515 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:13.129216 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:13.129129 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:13.297711 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:13.297675 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:19.748657 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:19.748622 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:19.855651 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:19.855622 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:26.657835 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:26.657798 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:27.121867 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:27.121838 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:27.446495 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:27.446388 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:30.271384 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.271351 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:30.401055 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.401023 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:30.598042 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.597961 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx"] Jul 06 19:35:30.598365 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.598352 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b55ca23a-5320-4054-9e73-bf0523c517ba" containerName="maas-api" Jul 06 19:35:30.598419 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.598367 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b55ca23a-5320-4054-9e73-bf0523c517ba" containerName="maas-api" Jul 06 19:35:30.598510 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.598428 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b55ca23a-5320-4054-9e73-bf0523c517ba" containerName="maas-api" Jul 06 19:35:30.601401 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.601383 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" Jul 06 19:35:30.604982 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.604962 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-3ed9b9ce-serving-cert\"" Jul 06 19:35:30.605829 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.605811 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-zxgdj\"" Jul 06 19:35:30.627547 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.627516 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx"] Jul 06 19:35:30.652518 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.652488 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:30.672761 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.672731 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5610867-41dd-4b09-90d1-946edd1a5d33-maas-api-tls\") pod \"maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx\" (UID: \"e5610867-41dd-4b09-90d1-946edd1a5d33\") " pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" Jul 06 19:35:30.672903 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.672793 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-955zw\" (UniqueName: \"kubernetes.io/projected/e5610867-41dd-4b09-90d1-946edd1a5d33-kube-api-access-955zw\") pod \"maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx\" (UID: \"e5610867-41dd-4b09-90d1-946edd1a5d33\") " pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" Jul 06 19:35:30.672965 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.672902 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5610867-41dd-4b09-90d1-946edd1a5d33-openshift-service-ca\") pod \"maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx\" (UID: \"e5610867-41dd-4b09-90d1-946edd1a5d33\") " pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" Jul 06 19:35:30.773850 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.773815 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5610867-41dd-4b09-90d1-946edd1a5d33-openshift-service-ca\") pod \"maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx\" (UID: \"e5610867-41dd-4b09-90d1-946edd1a5d33\") " pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" Jul 06 19:35:30.774012 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.773862 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5610867-41dd-4b09-90d1-946edd1a5d33-maas-api-tls\") pod \"maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx\" (UID: \"e5610867-41dd-4b09-90d1-946edd1a5d33\") " pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" Jul 06 19:35:30.774012 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.773884 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-955zw\" (UniqueName: \"kubernetes.io/projected/e5610867-41dd-4b09-90d1-946edd1a5d33-kube-api-access-955zw\") pod \"maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx\" (UID: \"e5610867-41dd-4b09-90d1-946edd1a5d33\") " pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" Jul 06 19:35:30.774469 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.774426 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5610867-41dd-4b09-90d1-946edd1a5d33-openshift-service-ca\") pod \"maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx\" (UID: \"e5610867-41dd-4b09-90d1-946edd1a5d33\") " pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" Jul 06 19:35:30.776289 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.776269 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5610867-41dd-4b09-90d1-946edd1a5d33-maas-api-tls\") pod \"maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx\" (UID: \"e5610867-41dd-4b09-90d1-946edd1a5d33\") " pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" Jul 06 19:35:30.792658 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.792626 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-955zw\" (UniqueName: \"kubernetes.io/projected/e5610867-41dd-4b09-90d1-946edd1a5d33-kube-api-access-955zw\") pod \"maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx\" (UID: \"e5610867-41dd-4b09-90d1-946edd1a5d33\") " pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" Jul 06 19:35:30.800355 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.800333 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:30.911504 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:30.911394 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" Jul 06 19:35:31.064301 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:31.064263 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx"] Jul 06 19:35:31.067943 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:35:31.067917 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5610867_41dd_4b09_90d1_946edd1a5d33.slice/crio-deba2767abc038ea74ab9fef6297a83becf7403dbb29aae6232446b8b64bbb1b WatchSource:0}: Error finding container deba2767abc038ea74ab9fef6297a83becf7403dbb29aae6232446b8b64bbb1b: Status 404 returned error can't find the container with id deba2767abc038ea74ab9fef6297a83becf7403dbb29aae6232446b8b64bbb1b Jul 06 19:35:31.365279 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:31.365240 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:31.465011 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:31.464989 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:31.947505 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:31.947469 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" event={"ID":"e5610867-41dd-4b09-90d1-946edd1a5d33","Type":"ContainerStarted","Data":"cc0ec80cc5e8781c9c04b03f5773c8c2d9e77a2636566ca8723fd7115234c245"} Jul 06 19:35:31.947725 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:31.947528 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" event={"ID":"e5610867-41dd-4b09-90d1-946edd1a5d33","Type":"ContainerStarted","Data":"deba2767abc038ea74ab9fef6297a83becf7403dbb29aae6232446b8b64bbb1b"} Jul 06 19:35:31.947725 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:31.947559 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" Jul 06 19:35:31.984843 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:31.984792 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" podStartSLOduration=1.64412447 podStartE2EDuration="1.98477713s" totalImagesPullingTime="340.65266ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:35:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:35:31.984088241 +0000 UTC m=+2172.345252034" watchObservedRunningTime="2026-07-06 19:35:31.98477713 +0000 UTC m=+2172.345940951" Jul 06 19:35:34.834534 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:34.834496 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:37.959026 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:37.958994 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" Jul 06 19:35:55.029105 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:35:55.029057 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:17.536870 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:17.536833 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:17.884507 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:17.884406 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx"] Jul 06 19:36:17.884671 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:17.884623 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" podUID="e5610867-41dd-4b09-90d1-946edd1a5d33" containerName="maas-api" containerID="cri-o://cc0ec80cc5e8781c9c04b03f5773c8c2d9e77a2636566ca8723fd7115234c245" gracePeriod=30 Jul 06 19:36:17.930207 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:17.930175 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:17.954431 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:17.954398 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" podUID="e5610867-41dd-4b09-90d1-946edd1a5d33" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.39:8443/health\": dial tcp 10.134.0.39:8443: connect: connection refused" Jul 06 19:36:18.131678 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.131654 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" Jul 06 19:36:18.174280 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.174196 2557 generic.go:356] "Generic (PLEG): container finished" podID="e5610867-41dd-4b09-90d1-946edd1a5d33" containerID="cc0ec80cc5e8781c9c04b03f5773c8c2d9e77a2636566ca8723fd7115234c245" exitCode=0 Jul 06 19:36:18.174280 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.174248 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" Jul 06 19:36:18.174477 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.174290 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" event={"ID":"e5610867-41dd-4b09-90d1-946edd1a5d33","Type":"ContainerDied","Data":"cc0ec80cc5e8781c9c04b03f5773c8c2d9e77a2636566ca8723fd7115234c245"} Jul 06 19:36:18.174477 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.174349 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx" event={"ID":"e5610867-41dd-4b09-90d1-946edd1a5d33","Type":"ContainerDied","Data":"deba2767abc038ea74ab9fef6297a83becf7403dbb29aae6232446b8b64bbb1b"} Jul 06 19:36:18.174477 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.174371 2557 scope.go:117] "RemoveContainer" containerID="cc0ec80cc5e8781c9c04b03f5773c8c2d9e77a2636566ca8723fd7115234c245" Jul 06 19:36:18.183031 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.183001 2557 scope.go:117] "RemoveContainer" containerID="cc0ec80cc5e8781c9c04b03f5773c8c2d9e77a2636566ca8723fd7115234c245" Jul 06 19:36:18.183288 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:36:18.183267 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc0ec80cc5e8781c9c04b03f5773c8c2d9e77a2636566ca8723fd7115234c245\": container with ID starting with cc0ec80cc5e8781c9c04b03f5773c8c2d9e77a2636566ca8723fd7115234c245 not found: ID does not exist" containerID="cc0ec80cc5e8781c9c04b03f5773c8c2d9e77a2636566ca8723fd7115234c245" Jul 06 19:36:18.183346 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.183295 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc0ec80cc5e8781c9c04b03f5773c8c2d9e77a2636566ca8723fd7115234c245"} err="failed to get container status \"cc0ec80cc5e8781c9c04b03f5773c8c2d9e77a2636566ca8723fd7115234c245\": rpc error: code = NotFound desc = could not find container \"cc0ec80cc5e8781c9c04b03f5773c8c2d9e77a2636566ca8723fd7115234c245\": container with ID starting with cc0ec80cc5e8781c9c04b03f5773c8c2d9e77a2636566ca8723fd7115234c245 not found: ID does not exist" Jul 06 19:36:18.209676 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.209647 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5610867-41dd-4b09-90d1-946edd1a5d33-maas-api-tls\") pod \"e5610867-41dd-4b09-90d1-946edd1a5d33\" (UID: \"e5610867-41dd-4b09-90d1-946edd1a5d33\") " Jul 06 19:36:18.209859 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.209704 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5610867-41dd-4b09-90d1-946edd1a5d33-openshift-service-ca\") pod \"e5610867-41dd-4b09-90d1-946edd1a5d33\" (UID: \"e5610867-41dd-4b09-90d1-946edd1a5d33\") " Jul 06 19:36:18.209859 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.209727 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-955zw\" (UniqueName: \"kubernetes.io/projected/e5610867-41dd-4b09-90d1-946edd1a5d33-kube-api-access-955zw\") pod \"e5610867-41dd-4b09-90d1-946edd1a5d33\" (UID: \"e5610867-41dd-4b09-90d1-946edd1a5d33\") " Jul 06 19:36:18.210129 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.210103 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5610867-41dd-4b09-90d1-946edd1a5d33-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e5610867-41dd-4b09-90d1-946edd1a5d33" (UID: "e5610867-41dd-4b09-90d1-946edd1a5d33"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:36:18.211881 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.211857 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5610867-41dd-4b09-90d1-946edd1a5d33-kube-api-access-955zw" (OuterVolumeSpecName: "kube-api-access-955zw") pod "e5610867-41dd-4b09-90d1-946edd1a5d33" (UID: "e5610867-41dd-4b09-90d1-946edd1a5d33"). InnerVolumeSpecName "kube-api-access-955zw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:36:18.211961 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.211861 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5610867-41dd-4b09-90d1-946edd1a5d33-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e5610867-41dd-4b09-90d1-946edd1a5d33" (UID: "e5610867-41dd-4b09-90d1-946edd1a5d33"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:36:18.310678 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.310644 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5610867-41dd-4b09-90d1-946edd1a5d33-maas-api-tls\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:36:18.310678 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.310674 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5610867-41dd-4b09-90d1-946edd1a5d33-openshift-service-ca\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:36:18.310678 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.310683 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-955zw\" (UniqueName: \"kubernetes.io/projected/e5610867-41dd-4b09-90d1-946edd1a5d33-kube-api-access-955zw\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:36:18.498754 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.498722 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx"] Jul 06 19:36:18.502484 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:18.502457 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-3ed9b9ce-856bf65c5b-s49wx"] Jul 06 19:36:20.203412 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:20.203381 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5610867-41dd-4b09-90d1-946edd1a5d33" path="/var/lib/kubelet/pods/e5610867-41dd-4b09-90d1-946edd1a5d33/volumes" Jul 06 19:36:22.389081 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:22.389047 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:23.616622 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:23.616596 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:28.665138 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:28.665101 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:41.228908 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:41.228870 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:41.318882 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:36:41.318798 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:00.314267 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:00.314231 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:01.060190 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:01.060159 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:07.153169 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:07.153136 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:32.416375 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:32.416345 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:32.522613 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:32.522580 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:32.784188 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:32.784150 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:33.007144 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:33.007112 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:34.338451 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:34.338412 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:34.604574 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:34.604496 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:35.373570 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:35.373541 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:35.531267 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:35.531236 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:37.036671 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:37.036639 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:43.269131 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:43.269096 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:43.354755 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:43.354720 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:43.967466 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:43.967419 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:50.303415 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:50.303378 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:50.428790 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:50.428757 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:50.553166 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:37:50.553133 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:01.079952 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:01.079914 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:02.539644 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:02.539607 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:22.425413 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:22.425326 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:29.010829 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:29.010792 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:29.092238 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:29.092206 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:29.366143 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:29.366062 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:29.828543 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:29.828474 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:29.933458 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:29.933401 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:33.290862 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.290832 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:33.478519 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.478482 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr"] Jul 06 19:38:33.478905 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.478893 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5610867-41dd-4b09-90d1-946edd1a5d33" containerName="maas-api" Jul 06 19:38:33.478949 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.478907 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5610867-41dd-4b09-90d1-946edd1a5d33" containerName="maas-api" Jul 06 19:38:33.478991 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.478981 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5610867-41dd-4b09-90d1-946edd1a5d33" containerName="maas-api" Jul 06 19:38:33.482205 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.482187 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" Jul 06 19:38:33.486245 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.486222 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-3d94e6-serving-cert\"" Jul 06 19:38:33.486368 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.486279 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-zxgdj\"" Jul 06 19:38:33.520321 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.520290 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr"] Jul 06 19:38:33.520949 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.520922 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:33.573256 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.573168 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b10bf9ce-0b6d-42be-8e75-8328b5642272-maas-api-tls\") pod \"maas-api-e2e-inf-a-3d94e6-95c96799c-982pr\" (UID: \"b10bf9ce-0b6d-42be-8e75-8328b5642272\") " pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" Jul 06 19:38:33.573256 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.573232 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sv4b\" (UniqueName: \"kubernetes.io/projected/b10bf9ce-0b6d-42be-8e75-8328b5642272-kube-api-access-8sv4b\") pod \"maas-api-e2e-inf-a-3d94e6-95c96799c-982pr\" (UID: \"b10bf9ce-0b6d-42be-8e75-8328b5642272\") " pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" Jul 06 19:38:33.573490 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.573373 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b10bf9ce-0b6d-42be-8e75-8328b5642272-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3d94e6-95c96799c-982pr\" (UID: \"b10bf9ce-0b6d-42be-8e75-8328b5642272\") " pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" Jul 06 19:38:33.673873 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.673838 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b10bf9ce-0b6d-42be-8e75-8328b5642272-maas-api-tls\") pod \"maas-api-e2e-inf-a-3d94e6-95c96799c-982pr\" (UID: \"b10bf9ce-0b6d-42be-8e75-8328b5642272\") " pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" Jul 06 19:38:33.674055 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.673883 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sv4b\" (UniqueName: \"kubernetes.io/projected/b10bf9ce-0b6d-42be-8e75-8328b5642272-kube-api-access-8sv4b\") pod \"maas-api-e2e-inf-a-3d94e6-95c96799c-982pr\" (UID: \"b10bf9ce-0b6d-42be-8e75-8328b5642272\") " pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" Jul 06 19:38:33.674055 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.673950 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b10bf9ce-0b6d-42be-8e75-8328b5642272-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3d94e6-95c96799c-982pr\" (UID: \"b10bf9ce-0b6d-42be-8e75-8328b5642272\") " pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" Jul 06 19:38:33.674055 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:38:33.673990 2557 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-3d94e6-serving-cert: secret "maas-api-e2e-inf-a-3d94e6-serving-cert" not found Jul 06 19:38:33.674165 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:38:33.674060 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10bf9ce-0b6d-42be-8e75-8328b5642272-maas-api-tls podName:b10bf9ce-0b6d-42be-8e75-8328b5642272 nodeName:}" failed. No retries permitted until 2026-07-06 19:38:34.174042606 +0000 UTC m=+2354.535206377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b10bf9ce-0b6d-42be-8e75-8328b5642272-maas-api-tls") pod "maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" (UID: "b10bf9ce-0b6d-42be-8e75-8328b5642272") : secret "maas-api-e2e-inf-a-3d94e6-serving-cert" not found Jul 06 19:38:33.674628 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.674609 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b10bf9ce-0b6d-42be-8e75-8328b5642272-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3d94e6-95c96799c-982pr\" (UID: \"b10bf9ce-0b6d-42be-8e75-8328b5642272\") " pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" Jul 06 19:38:33.695450 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.695411 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sv4b\" (UniqueName: \"kubernetes.io/projected/b10bf9ce-0b6d-42be-8e75-8328b5642272-kube-api-access-8sv4b\") pod \"maas-api-e2e-inf-a-3d94e6-95c96799c-982pr\" (UID: \"b10bf9ce-0b6d-42be-8e75-8328b5642272\") " pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" Jul 06 19:38:33.747049 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.747015 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:33.875704 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:33.875599 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:34.178716 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:34.178616 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b10bf9ce-0b6d-42be-8e75-8328b5642272-maas-api-tls\") pod \"maas-api-e2e-inf-a-3d94e6-95c96799c-982pr\" (UID: \"b10bf9ce-0b6d-42be-8e75-8328b5642272\") " pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" Jul 06 19:38:34.181114 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:34.181087 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b10bf9ce-0b6d-42be-8e75-8328b5642272-maas-api-tls\") pod \"maas-api-e2e-inf-a-3d94e6-95c96799c-982pr\" (UID: \"b10bf9ce-0b6d-42be-8e75-8328b5642272\") " pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" Jul 06 19:38:34.392817 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:34.392784 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" Jul 06 19:38:34.400279 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:34.400256 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:34.518377 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:34.518346 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:34.546530 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:34.546496 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr"] Jul 06 19:38:34.552336 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:38:34.552302 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb10bf9ce_0b6d_42be_8e75_8328b5642272.slice/crio-eab56ca4ee5d73b40b94aaad50b93b824ee4c91f9d519ef3226ab74a0cbfc8aa WatchSource:0}: Error finding container eab56ca4ee5d73b40b94aaad50b93b824ee4c91f9d519ef3226ab74a0cbfc8aa: Status 404 returned error can't find the container with id eab56ca4ee5d73b40b94aaad50b93b824ee4c91f9d519ef3226ab74a0cbfc8aa Jul 06 19:38:34.553776 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:34.553757 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:38:34.842553 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:34.842512 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" event={"ID":"b10bf9ce-0b6d-42be-8e75-8328b5642272","Type":"ContainerStarted","Data":"eab56ca4ee5d73b40b94aaad50b93b824ee4c91f9d519ef3226ab74a0cbfc8aa"} Jul 06 19:38:35.849893 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:35.849857 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" event={"ID":"b10bf9ce-0b6d-42be-8e75-8328b5642272","Type":"ContainerStarted","Data":"36e60b4c43a5c0c228f1b46a3724a7380f5535e8041f62cfb1e3b51629caedb8"} Jul 06 19:38:35.850320 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:35.849969 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" Jul 06 19:38:35.887706 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:35.887657 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" podStartSLOduration=2.439199987 podStartE2EDuration="2.887639624s" totalImagesPullingTime="448.439637ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:38:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:38:35.886110494 +0000 UTC m=+2356.247274287" watchObservedRunningTime="2026-07-06 19:38:35.887639624 +0000 UTC m=+2356.248803417" Jul 06 19:38:37.805478 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:37.805417 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:38.180462 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:38.180345 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:38.703663 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:38.703633 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:38.959787 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:38.959697 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:41.338061 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:41.338025 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:41.432888 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:41.432859 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:41.512138 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:41.512094 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr"] Jul 06 19:38:41.515970 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:41.515948 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" Jul 06 19:38:41.518992 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:41.518966 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-7c8a24-serving-cert\"" Jul 06 19:38:41.530459 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:41.528256 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr"] Jul 06 19:38:41.548232 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:41.548201 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frvfl\" (UniqueName: \"kubernetes.io/projected/27417d1a-3699-4e41-92d8-0768f7b7c904-kube-api-access-frvfl\") pod \"maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr\" (UID: \"27417d1a-3699-4e41-92d8-0768f7b7c904\") " pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" Jul 06 19:38:41.548398 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:41.548326 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27417d1a-3699-4e41-92d8-0768f7b7c904-maas-api-tls\") pod \"maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr\" (UID: \"27417d1a-3699-4e41-92d8-0768f7b7c904\") " pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" Jul 06 19:38:41.548398 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:41.548357 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27417d1a-3699-4e41-92d8-0768f7b7c904-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr\" (UID: \"27417d1a-3699-4e41-92d8-0768f7b7c904\") " pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" Jul 06 19:38:41.557020 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:41.556993 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:41.649377 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:41.649278 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27417d1a-3699-4e41-92d8-0768f7b7c904-maas-api-tls\") pod \"maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr\" (UID: \"27417d1a-3699-4e41-92d8-0768f7b7c904\") " pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" Jul 06 19:38:41.649377 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:41.649322 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27417d1a-3699-4e41-92d8-0768f7b7c904-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr\" (UID: \"27417d1a-3699-4e41-92d8-0768f7b7c904\") " pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" Jul 06 19:38:41.649810 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:41.649382 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frvfl\" (UniqueName: \"kubernetes.io/projected/27417d1a-3699-4e41-92d8-0768f7b7c904-kube-api-access-frvfl\") pod \"maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr\" (UID: \"27417d1a-3699-4e41-92d8-0768f7b7c904\") " pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" Jul 06 19:38:41.649810 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:38:41.649474 2557 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-7c8a24-serving-cert: secret "maas-api-e2e-inf-b-7c8a24-serving-cert" not found Jul 06 19:38:41.649810 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:38:41.649570 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27417d1a-3699-4e41-92d8-0768f7b7c904-maas-api-tls podName:27417d1a-3699-4e41-92d8-0768f7b7c904 nodeName:}" failed. No retries permitted until 2026-07-06 19:38:42.149548612 +0000 UTC m=+2362.510712396 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/27417d1a-3699-4e41-92d8-0768f7b7c904-maas-api-tls") pod "maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" (UID: "27417d1a-3699-4e41-92d8-0768f7b7c904") : secret "maas-api-e2e-inf-b-7c8a24-serving-cert" not found Jul 06 19:38:41.650208 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:41.650184 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27417d1a-3699-4e41-92d8-0768f7b7c904-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr\" (UID: \"27417d1a-3699-4e41-92d8-0768f7b7c904\") " pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" Jul 06 19:38:41.668409 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:41.668374 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frvfl\" (UniqueName: \"kubernetes.io/projected/27417d1a-3699-4e41-92d8-0768f7b7c904-kube-api-access-frvfl\") pod \"maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr\" (UID: \"27417d1a-3699-4e41-92d8-0768f7b7c904\") " pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" Jul 06 19:38:41.861036 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:41.861009 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" Jul 06 19:38:42.074186 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:42.074152 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:42.154151 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:42.154115 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27417d1a-3699-4e41-92d8-0768f7b7c904-maas-api-tls\") pod \"maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr\" (UID: \"27417d1a-3699-4e41-92d8-0768f7b7c904\") " pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" Jul 06 19:38:42.156561 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:42.156538 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27417d1a-3699-4e41-92d8-0768f7b7c904-maas-api-tls\") pod \"maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr\" (UID: \"27417d1a-3699-4e41-92d8-0768f7b7c904\") " pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" Jul 06 19:38:42.167088 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:42.167062 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:42.429902 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:42.429796 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" Jul 06 19:38:42.573831 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:42.573805 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr"] Jul 06 19:38:42.576413 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:38:42.576385 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27417d1a_3699_4e41_92d8_0768f7b7c904.slice/crio-20989c327305c79eab0e509b05d877a6d5475e676ce068d534a8864787489ffc WatchSource:0}: Error finding container 20989c327305c79eab0e509b05d877a6d5475e676ce068d534a8864787489ffc: Status 404 returned error can't find the container with id 20989c327305c79eab0e509b05d877a6d5475e676ce068d534a8864787489ffc Jul 06 19:38:42.885242 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:42.885200 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" event={"ID":"27417d1a-3699-4e41-92d8-0768f7b7c904","Type":"ContainerStarted","Data":"20989c327305c79eab0e509b05d877a6d5475e676ce068d534a8864787489ffc"} Jul 06 19:38:43.891112 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:43.891063 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" event={"ID":"27417d1a-3699-4e41-92d8-0768f7b7c904","Type":"ContainerStarted","Data":"4635d24c7e74993e1eed3dc2321c1bccb9cca7c8464666049622a2277ca12268"} Jul 06 19:38:43.891549 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:43.891242 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" Jul 06 19:38:43.910732 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:43.910684 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" podStartSLOduration=2.569776055 podStartE2EDuration="2.910666965s" totalImagesPullingTime="340.89091ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:38:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:38:43.909942034 +0000 UTC m=+2364.271105826" watchObservedRunningTime="2026-07-06 19:38:43.910666965 +0000 UTC m=+2364.271830793" Jul 06 19:38:45.785055 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:45.785014 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:49.902010 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:49.901981 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" Jul 06 19:38:51.829556 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:51.829513 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:55.424212 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:55.424179 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:55.555111 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:38:55.555078 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:05.305955 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:05.305921 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:05.407893 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:05.407862 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:07.962926 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:07.962892 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:25.270985 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:25.270948 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:31.652617 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:31.652580 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:31.754568 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:31.754531 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:47.174748 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:47.174673 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:48.008150 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:48.008112 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:48.197953 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:48.197920 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:48.662292 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:48.662258 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:48.764076 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:48.764039 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:48.863833 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:48.863798 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:48.986513 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:48.986392 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr"] Jul 06 19:39:48.986707 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:48.986673 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" podUID="b10bf9ce-0b6d-42be-8e75-8328b5642272" containerName="maas-api" containerID="cri-o://36e60b4c43a5c0c228f1b46a3724a7380f5535e8041f62cfb1e3b51629caedb8" gracePeriod=30 Jul 06 19:39:49.021560 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:49.021532 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:49.219747 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:49.219709 2557 generic.go:356] "Generic (PLEG): container finished" podID="b10bf9ce-0b6d-42be-8e75-8328b5642272" containerID="36e60b4c43a5c0c228f1b46a3724a7380f5535e8041f62cfb1e3b51629caedb8" exitCode=0 Jul 06 19:39:49.220073 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:49.219812 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" event={"ID":"b10bf9ce-0b6d-42be-8e75-8328b5642272","Type":"ContainerDied","Data":"36e60b4c43a5c0c228f1b46a3724a7380f5535e8041f62cfb1e3b51629caedb8"} Jul 06 19:39:49.240001 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:49.239945 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" Jul 06 19:39:49.380516 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:49.380479 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sv4b\" (UniqueName: \"kubernetes.io/projected/b10bf9ce-0b6d-42be-8e75-8328b5642272-kube-api-access-8sv4b\") pod \"b10bf9ce-0b6d-42be-8e75-8328b5642272\" (UID: \"b10bf9ce-0b6d-42be-8e75-8328b5642272\") " Jul 06 19:39:49.380719 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:49.380547 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b10bf9ce-0b6d-42be-8e75-8328b5642272-openshift-service-ca\") pod \"b10bf9ce-0b6d-42be-8e75-8328b5642272\" (UID: \"b10bf9ce-0b6d-42be-8e75-8328b5642272\") " Jul 06 19:39:49.380719 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:49.380575 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b10bf9ce-0b6d-42be-8e75-8328b5642272-maas-api-tls\") pod \"b10bf9ce-0b6d-42be-8e75-8328b5642272\" (UID: \"b10bf9ce-0b6d-42be-8e75-8328b5642272\") " Jul 06 19:39:49.380856 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:49.380827 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b10bf9ce-0b6d-42be-8e75-8328b5642272-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b10bf9ce-0b6d-42be-8e75-8328b5642272" (UID: "b10bf9ce-0b6d-42be-8e75-8328b5642272"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:39:49.380944 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:49.380928 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b10bf9ce-0b6d-42be-8e75-8328b5642272-openshift-service-ca\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:39:49.382689 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:49.382670 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b10bf9ce-0b6d-42be-8e75-8328b5642272-kube-api-access-8sv4b" (OuterVolumeSpecName: "kube-api-access-8sv4b") pod "b10bf9ce-0b6d-42be-8e75-8328b5642272" (UID: "b10bf9ce-0b6d-42be-8e75-8328b5642272"). InnerVolumeSpecName "kube-api-access-8sv4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:39:49.382744 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:49.382694 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b10bf9ce-0b6d-42be-8e75-8328b5642272-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b10bf9ce-0b6d-42be-8e75-8328b5642272" (UID: "b10bf9ce-0b6d-42be-8e75-8328b5642272"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:39:49.482338 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:49.482309 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sv4b\" (UniqueName: \"kubernetes.io/projected/b10bf9ce-0b6d-42be-8e75-8328b5642272-kube-api-access-8sv4b\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:39:49.482338 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:49.482335 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b10bf9ce-0b6d-42be-8e75-8328b5642272-maas-api-tls\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:39:50.225834 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:50.225813 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" Jul 06 19:39:50.225834 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:50.225824 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr" event={"ID":"b10bf9ce-0b6d-42be-8e75-8328b5642272","Type":"ContainerDied","Data":"eab56ca4ee5d73b40b94aaad50b93b824ee4c91f9d519ef3226ab74a0cbfc8aa"} Jul 06 19:39:50.226301 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:50.225884 2557 scope.go:117] "RemoveContainer" containerID="36e60b4c43a5c0c228f1b46a3724a7380f5535e8041f62cfb1e3b51629caedb8" Jul 06 19:39:50.261162 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:50.261134 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr"] Jul 06 19:39:50.268374 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:50.268350 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-3d94e6-95c96799c-982pr"] Jul 06 19:39:51.381343 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:51.381309 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:52.198027 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:52.197993 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b10bf9ce-0b6d-42be-8e75-8328b5642272" path="/var/lib/kubelet/pods/b10bf9ce-0b6d-42be-8e75-8328b5642272/volumes" Jul 06 19:39:53.739651 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:39:53.739619 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:03.746273 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:40:03.746237 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:11.051973 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:40:11.051938 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:16.101920 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:40:16.101882 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:37.479128 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:40:37.479091 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:38.021049 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:40:38.021020 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:40.971140 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:40:40.971092 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:46.169998 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:40:46.169957 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:51.166299 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:40:51.166259 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:51.283026 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:40:51.282969 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:53.021978 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:40:53.021933 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:53.201050 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:40:53.200916 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:56.133309 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:40:56.133265 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:56.366410 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:40:56.366378 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:58.969688 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:40:58.969655 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:01.119614 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:01.119564 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:01.270735 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:01.270698 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:01.946422 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:01.946357 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:02.065045 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:02.064999 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:03.608703 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:03.608672 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:06.106093 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:06.106053 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:06.252680 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:06.252641 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:16.175227 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:16.175119 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:16.367005 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:16.366964 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:17.358139 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:17.358086 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:19.115635 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:19.115605 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:29.992929 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:29.992889 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:30.357385 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:30.357280 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:30.956169 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:30.956134 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:32.206910 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.206840 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:32.428354 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.428325 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr"] Jul 06 19:41:32.428596 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.428569 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" podUID="27417d1a-3699-4e41-92d8-0768f7b7c904" containerName="maas-api" containerID="cri-o://4635d24c7e74993e1eed3dc2321c1bccb9cca7c8464666049622a2277ca12268" gracePeriod=30 Jul 06 19:41:32.492908 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.492868 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:32.673039 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.673015 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" Jul 06 19:41:32.673740 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.673721 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:32.734400 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.734368 2557 generic.go:356] "Generic (PLEG): container finished" podID="27417d1a-3699-4e41-92d8-0768f7b7c904" containerID="4635d24c7e74993e1eed3dc2321c1bccb9cca7c8464666049622a2277ca12268" exitCode=0 Jul 06 19:41:32.734608 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.734409 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" Jul 06 19:41:32.734608 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.734469 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" event={"ID":"27417d1a-3699-4e41-92d8-0768f7b7c904","Type":"ContainerDied","Data":"4635d24c7e74993e1eed3dc2321c1bccb9cca7c8464666049622a2277ca12268"} Jul 06 19:41:32.734608 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.734521 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr" event={"ID":"27417d1a-3699-4e41-92d8-0768f7b7c904","Type":"ContainerDied","Data":"20989c327305c79eab0e509b05d877a6d5475e676ce068d534a8864787489ffc"} Jul 06 19:41:32.734608 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.734552 2557 scope.go:117] "RemoveContainer" containerID="4635d24c7e74993e1eed3dc2321c1bccb9cca7c8464666049622a2277ca12268" Jul 06 19:41:32.743106 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.743088 2557 scope.go:117] "RemoveContainer" containerID="4635d24c7e74993e1eed3dc2321c1bccb9cca7c8464666049622a2277ca12268" Jul 06 19:41:32.743387 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:41:32.743366 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4635d24c7e74993e1eed3dc2321c1bccb9cca7c8464666049622a2277ca12268\": container with ID starting with 4635d24c7e74993e1eed3dc2321c1bccb9cca7c8464666049622a2277ca12268 not found: ID does not exist" containerID="4635d24c7e74993e1eed3dc2321c1bccb9cca7c8464666049622a2277ca12268" Jul 06 19:41:32.743517 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.743400 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4635d24c7e74993e1eed3dc2321c1bccb9cca7c8464666049622a2277ca12268"} err="failed to get container status \"4635d24c7e74993e1eed3dc2321c1bccb9cca7c8464666049622a2277ca12268\": rpc error: code = NotFound desc = could not find container \"4635d24c7e74993e1eed3dc2321c1bccb9cca7c8464666049622a2277ca12268\": container with ID starting with 4635d24c7e74993e1eed3dc2321c1bccb9cca7c8464666049622a2277ca12268 not found: ID does not exist" Jul 06 19:41:32.744659 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.744637 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frvfl\" (UniqueName: \"kubernetes.io/projected/27417d1a-3699-4e41-92d8-0768f7b7c904-kube-api-access-frvfl\") pod \"27417d1a-3699-4e41-92d8-0768f7b7c904\" (UID: \"27417d1a-3699-4e41-92d8-0768f7b7c904\") " Jul 06 19:41:32.744823 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.744809 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27417d1a-3699-4e41-92d8-0768f7b7c904-openshift-service-ca\") pod \"27417d1a-3699-4e41-92d8-0768f7b7c904\" (UID: \"27417d1a-3699-4e41-92d8-0768f7b7c904\") " Jul 06 19:41:32.744898 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.744834 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27417d1a-3699-4e41-92d8-0768f7b7c904-maas-api-tls\") pod \"27417d1a-3699-4e41-92d8-0768f7b7c904\" (UID: \"27417d1a-3699-4e41-92d8-0768f7b7c904\") " Jul 06 19:41:32.745183 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.745160 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27417d1a-3699-4e41-92d8-0768f7b7c904-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "27417d1a-3699-4e41-92d8-0768f7b7c904" (UID: "27417d1a-3699-4e41-92d8-0768f7b7c904"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:41:32.746636 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.746614 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27417d1a-3699-4e41-92d8-0768f7b7c904-kube-api-access-frvfl" (OuterVolumeSpecName: "kube-api-access-frvfl") pod "27417d1a-3699-4e41-92d8-0768f7b7c904" (UID: "27417d1a-3699-4e41-92d8-0768f7b7c904"). InnerVolumeSpecName "kube-api-access-frvfl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:41:32.746753 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.746733 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27417d1a-3699-4e41-92d8-0768f7b7c904-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "27417d1a-3699-4e41-92d8-0768f7b7c904" (UID: "27417d1a-3699-4e41-92d8-0768f7b7c904"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:41:32.845992 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.845956 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27417d1a-3699-4e41-92d8-0768f7b7c904-openshift-service-ca\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:41:32.845992 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.845986 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27417d1a-3699-4e41-92d8-0768f7b7c904-maas-api-tls\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:41:32.845992 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:32.845996 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frvfl\" (UniqueName: \"kubernetes.io/projected/27417d1a-3699-4e41-92d8-0768f7b7c904-kube-api-access-frvfl\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:41:33.060417 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:33.060382 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr"] Jul 06 19:41:33.063079 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:33.063051 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c8a24-84979f5778-ph5lr"] Jul 06 19:41:33.716630 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:33.716599 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:34.196775 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:34.196734 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27417d1a-3699-4e41-92d8-0768f7b7c904" path="/var/lib/kubelet/pods/27417d1a-3699-4e41-92d8-0768f7b7c904/volumes" Jul 06 19:41:38.376886 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:38.376845 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:39.210587 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:39.210551 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:40.045063 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:40.045030 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:46.229457 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:46.229398 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:47.842315 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:47.842276 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:48.704303 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:48.704263 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:50.121170 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:50.121134 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:51.279588 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:51.279549 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:54.892773 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:54.892735 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:55.777892 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:55.777857 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:56.652276 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:56.652246 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:57.064616 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:57.064585 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:57.576140 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:57.576106 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:57.752576 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:57.752542 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:57.942465 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:41:57.942360 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:02.463762 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:02.463733 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:03.841925 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:03.841896 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:04.503602 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:04.503575 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:05.198622 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:05.198586 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:05.348010 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:05.347965 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:05.495370 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:05.495340 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:07.736961 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:07.736921 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:07.870063 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:07.870032 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq"] Jul 06 19:42:07.870416 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:07.870404 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27417d1a-3699-4e41-92d8-0768f7b7c904" containerName="maas-api" Jul 06 19:42:07.870478 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:07.870417 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="27417d1a-3699-4e41-92d8-0768f7b7c904" containerName="maas-api" Jul 06 19:42:07.870478 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:07.870460 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b10bf9ce-0b6d-42be-8e75-8328b5642272" containerName="maas-api" Jul 06 19:42:07.870478 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:07.870466 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b10bf9ce-0b6d-42be-8e75-8328b5642272" containerName="maas-api" Jul 06 19:42:07.870574 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:07.870531 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b10bf9ce-0b6d-42be-8e75-8328b5642272" containerName="maas-api" Jul 06 19:42:07.870574 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:07.870546 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="27417d1a-3699-4e41-92d8-0768f7b7c904" containerName="maas-api" Jul 06 19:42:07.873735 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:07.873714 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" Jul 06 19:42:07.876919 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:07.876895 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-zxgdj\"" Jul 06 19:42:07.883272 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:07.883252 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-27c473-serving-cert\"" Jul 06 19:42:07.905706 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:07.905673 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq"] Jul 06 19:42:07.950008 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:07.949971 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:07.978887 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:07.978850 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b560fe3-a5b0-4a5c-895f-2565bde3f201-openshift-service-ca\") pod \"maas-api-e2e-api-a-27c473-8654549995-nxwzq\" (UID: \"9b560fe3-a5b0-4a5c-895f-2565bde3f201\") " pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" Jul 06 19:42:07.978887 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:07.978893 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phvw8\" (UniqueName: \"kubernetes.io/projected/9b560fe3-a5b0-4a5c-895f-2565bde3f201-kube-api-access-phvw8\") pod \"maas-api-e2e-api-a-27c473-8654549995-nxwzq\" (UID: \"9b560fe3-a5b0-4a5c-895f-2565bde3f201\") " pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" Jul 06 19:42:07.979110 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:07.978969 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b560fe3-a5b0-4a5c-895f-2565bde3f201-maas-api-tls\") pod \"maas-api-e2e-api-a-27c473-8654549995-nxwzq\" (UID: \"9b560fe3-a5b0-4a5c-895f-2565bde3f201\") " pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" Jul 06 19:42:08.080081 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:08.079988 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phvw8\" (UniqueName: \"kubernetes.io/projected/9b560fe3-a5b0-4a5c-895f-2565bde3f201-kube-api-access-phvw8\") pod \"maas-api-e2e-api-a-27c473-8654549995-nxwzq\" (UID: \"9b560fe3-a5b0-4a5c-895f-2565bde3f201\") " pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" Jul 06 19:42:08.080081 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:08.080035 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b560fe3-a5b0-4a5c-895f-2565bde3f201-maas-api-tls\") pod \"maas-api-e2e-api-a-27c473-8654549995-nxwzq\" (UID: \"9b560fe3-a5b0-4a5c-895f-2565bde3f201\") " pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" Jul 06 19:42:08.080318 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:08.080121 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b560fe3-a5b0-4a5c-895f-2565bde3f201-openshift-service-ca\") pod \"maas-api-e2e-api-a-27c473-8654549995-nxwzq\" (UID: \"9b560fe3-a5b0-4a5c-895f-2565bde3f201\") " pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" Jul 06 19:42:08.080318 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:42:08.080215 2557 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-27c473-serving-cert: secret "maas-api-e2e-api-a-27c473-serving-cert" not found Jul 06 19:42:08.080318 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:42:08.080292 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b560fe3-a5b0-4a5c-895f-2565bde3f201-maas-api-tls podName:9b560fe3-a5b0-4a5c-895f-2565bde3f201 nodeName:}" failed. No retries permitted until 2026-07-06 19:42:08.580270292 +0000 UTC m=+2568.941434080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9b560fe3-a5b0-4a5c-895f-2565bde3f201-maas-api-tls") pod "maas-api-e2e-api-a-27c473-8654549995-nxwzq" (UID: "9b560fe3-a5b0-4a5c-895f-2565bde3f201") : secret "maas-api-e2e-api-a-27c473-serving-cert" not found Jul 06 19:42:08.080801 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:08.080782 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b560fe3-a5b0-4a5c-895f-2565bde3f201-openshift-service-ca\") pod \"maas-api-e2e-api-a-27c473-8654549995-nxwzq\" (UID: \"9b560fe3-a5b0-4a5c-895f-2565bde3f201\") " pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" Jul 06 19:42:08.100254 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:08.100223 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:08.105404 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:08.105378 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phvw8\" (UniqueName: \"kubernetes.io/projected/9b560fe3-a5b0-4a5c-895f-2565bde3f201-kube-api-access-phvw8\") pod \"maas-api-e2e-api-a-27c473-8654549995-nxwzq\" (UID: \"9b560fe3-a5b0-4a5c-895f-2565bde3f201\") " pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" Jul 06 19:42:08.311796 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:08.311765 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:08.584485 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:08.584452 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b560fe3-a5b0-4a5c-895f-2565bde3f201-maas-api-tls\") pod \"maas-api-e2e-api-a-27c473-8654549995-nxwzq\" (UID: \"9b560fe3-a5b0-4a5c-895f-2565bde3f201\") " pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" Jul 06 19:42:08.586908 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:08.586881 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b560fe3-a5b0-4a5c-895f-2565bde3f201-maas-api-tls\") pod \"maas-api-e2e-api-a-27c473-8654549995-nxwzq\" (UID: \"9b560fe3-a5b0-4a5c-895f-2565bde3f201\") " pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" Jul 06 19:42:08.665292 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:08.665258 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:08.784665 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:08.784625 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" Jul 06 19:42:08.855163 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:08.855142 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:08.961388 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:08.961352 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq"] Jul 06 19:42:08.965058 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:42:08.965035 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b560fe3_a5b0_4a5c_895f_2565bde3f201.slice/crio-ba92393cf4f46f91d3c671a3cc381514e1cb9eea00c81b146ff76812e81f13da WatchSource:0}: Error finding container ba92393cf4f46f91d3c671a3cc381514e1cb9eea00c81b146ff76812e81f13da: Status 404 returned error can't find the container with id ba92393cf4f46f91d3c671a3cc381514e1cb9eea00c81b146ff76812e81f13da Jul 06 19:42:09.172342 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:09.172268 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:09.341920 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:09.341887 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:09.924311 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:09.924274 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" event={"ID":"9b560fe3-a5b0-4a5c-895f-2565bde3f201","Type":"ContainerStarted","Data":"18831d185195e9714251318baa0aafe7697e9fc8f40579d539cb3a4db7aefad7"} Jul 06 19:42:09.924723 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:09.924337 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" event={"ID":"9b560fe3-a5b0-4a5c-895f-2565bde3f201","Type":"ContainerStarted","Data":"ba92393cf4f46f91d3c671a3cc381514e1cb9eea00c81b146ff76812e81f13da"} Jul 06 19:42:09.924723 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:09.924422 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" Jul 06 19:42:11.886381 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:11.886323 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" podStartSLOduration=4.455319817 podStartE2EDuration="4.886305852s" totalImagesPullingTime="430.986035ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:42:11.877673021 +0000 UTC m=+2572.238836824" watchObservedRunningTime="2026-07-06 19:42:11.886305852 +0000 UTC m=+2572.247469646" Jul 06 19:42:12.709686 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:12.709652 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:15.934175 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:15.934145 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" Jul 06 19:42:24.484580 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:24.484549 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:35.279739 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:35.279702 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:37.618144 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:37.618108 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:37.963894 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:37.963802 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:38.659767 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:38.659733 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:38.999186 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:38.999154 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:40.381201 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:40.381172 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:41.334775 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:41.334700 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:42.048295 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:42.048259 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:42.512028 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:42.511991 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:42.739568 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:42.739535 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:43.227380 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:43.227345 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:43.544109 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:43.544075 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:44.295341 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:44.295307 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x"] Jul 06 19:42:44.299019 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:44.298998 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" Jul 06 19:42:44.304715 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:44.304691 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-dd3bb9-serving-cert\"" Jul 06 19:42:44.353565 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:44.353538 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:44.378134 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:44.378100 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x"] Jul 06 19:42:44.417575 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:44.417538 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6rb5\" (UniqueName: \"kubernetes.io/projected/6c81b264-30e9-4ddb-b483-eb29e6640b0f-kube-api-access-q6rb5\") pod \"maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x\" (UID: \"6c81b264-30e9-4ddb-b483-eb29e6640b0f\") " pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" Jul 06 19:42:44.417771 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:44.417643 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c81b264-30e9-4ddb-b483-eb29e6640b0f-openshift-service-ca\") pod \"maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x\" (UID: \"6c81b264-30e9-4ddb-b483-eb29e6640b0f\") " pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" Jul 06 19:42:44.417771 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:44.417671 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c81b264-30e9-4ddb-b483-eb29e6640b0f-maas-api-tls\") pod \"maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x\" (UID: \"6c81b264-30e9-4ddb-b483-eb29e6640b0f\") " pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" Jul 06 19:42:44.518405 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:44.518369 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c81b264-30e9-4ddb-b483-eb29e6640b0f-openshift-service-ca\") pod \"maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x\" (UID: \"6c81b264-30e9-4ddb-b483-eb29e6640b0f\") " pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" Jul 06 19:42:44.518405 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:44.518407 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c81b264-30e9-4ddb-b483-eb29e6640b0f-maas-api-tls\") pod \"maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x\" (UID: \"6c81b264-30e9-4ddb-b483-eb29e6640b0f\") " pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" Jul 06 19:42:44.518692 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:44.518514 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6rb5\" (UniqueName: \"kubernetes.io/projected/6c81b264-30e9-4ddb-b483-eb29e6640b0f-kube-api-access-q6rb5\") pod \"maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x\" (UID: \"6c81b264-30e9-4ddb-b483-eb29e6640b0f\") " pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" Jul 06 19:42:44.518692 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:42:44.518629 2557 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-dd3bb9-serving-cert: secret "maas-api-e2e-api-b-dd3bb9-serving-cert" not found Jul 06 19:42:44.518803 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:42:44.518714 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c81b264-30e9-4ddb-b483-eb29e6640b0f-maas-api-tls podName:6c81b264-30e9-4ddb-b483-eb29e6640b0f nodeName:}" failed. No retries permitted until 2026-07-06 19:42:45.018690657 +0000 UTC m=+2605.379854433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6c81b264-30e9-4ddb-b483-eb29e6640b0f-maas-api-tls") pod "maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" (UID: "6c81b264-30e9-4ddb-b483-eb29e6640b0f") : secret "maas-api-e2e-api-b-dd3bb9-serving-cert" not found Jul 06 19:42:44.519053 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:44.519033 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c81b264-30e9-4ddb-b483-eb29e6640b0f-openshift-service-ca\") pod \"maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x\" (UID: \"6c81b264-30e9-4ddb-b483-eb29e6640b0f\") " pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" Jul 06 19:42:44.581858 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:44.581782 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6rb5\" (UniqueName: \"kubernetes.io/projected/6c81b264-30e9-4ddb-b483-eb29e6640b0f-kube-api-access-q6rb5\") pod \"maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x\" (UID: \"6c81b264-30e9-4ddb-b483-eb29e6640b0f\") " pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" Jul 06 19:42:44.797426 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:44.797374 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:45.024482 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:45.024415 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c81b264-30e9-4ddb-b483-eb29e6640b0f-maas-api-tls\") pod \"maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x\" (UID: \"6c81b264-30e9-4ddb-b483-eb29e6640b0f\") " pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" Jul 06 19:42:45.026971 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:45.026935 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c81b264-30e9-4ddb-b483-eb29e6640b0f-maas-api-tls\") pod \"maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x\" (UID: \"6c81b264-30e9-4ddb-b483-eb29e6640b0f\") " pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" Jul 06 19:42:45.209607 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:45.209576 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" Jul 06 19:42:45.376572 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:45.376542 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x"] Jul 06 19:42:45.380086 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:42:45.380053 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c81b264_30e9_4ddb_b483_eb29e6640b0f.slice/crio-c65a9a1594fb0430cc95ee618d85465e413146adc5e158216ce7d7b93eb1f72b WatchSource:0}: Error finding container c65a9a1594fb0430cc95ee618d85465e413146adc5e158216ce7d7b93eb1f72b: Status 404 returned error can't find the container with id c65a9a1594fb0430cc95ee618d85465e413146adc5e158216ce7d7b93eb1f72b Jul 06 19:42:45.684901 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:45.684819 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:46.103899 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:46.103861 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" event={"ID":"6c81b264-30e9-4ddb-b483-eb29e6640b0f","Type":"ContainerStarted","Data":"3e5263b58af6f028c6e2d7d24733bd09229ebbcabbd115821a24424e9edba760"} Jul 06 19:42:46.104120 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:46.103920 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" event={"ID":"6c81b264-30e9-4ddb-b483-eb29e6640b0f","Type":"ContainerStarted","Data":"c65a9a1594fb0430cc95ee618d85465e413146adc5e158216ce7d7b93eb1f72b"} Jul 06 19:42:46.104120 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:46.103988 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" Jul 06 19:42:46.380541 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:46.375740 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" podStartSLOduration=2.01163922 podStartE2EDuration="2.375718835s" totalImagesPullingTime="364.079615ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:42:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:42:46.372373551 +0000 UTC m=+2606.733537342" watchObservedRunningTime="2026-07-06 19:42:46.375718835 +0000 UTC m=+2606.736882628" Jul 06 19:42:47.146559 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:47.146522 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:47.305205 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:47.305167 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:52.114567 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:52.114536 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" Jul 06 19:42:53.647370 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:53.647336 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:55.138301 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:55.138270 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:55.485870 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:55.485768 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:55.661577 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:55.661544 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:55.761030 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:55.760996 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:55.943237 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:55.943203 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq"] Jul 06 19:42:55.943457 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:55.943408 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" podUID="9b560fe3-a5b0-4a5c-895f-2565bde3f201" containerName="maas-api" containerID="cri-o://18831d185195e9714251318baa0aafe7697e9fc8f40579d539cb3a4db7aefad7" gracePeriod=30 Jul 06 19:42:55.967550 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:55.967507 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:56.155098 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:56.155066 2557 generic.go:356] "Generic (PLEG): container finished" podID="9b560fe3-a5b0-4a5c-895f-2565bde3f201" containerID="18831d185195e9714251318baa0aafe7697e9fc8f40579d539cb3a4db7aefad7" exitCode=0 Jul 06 19:42:56.155591 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:56.155123 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" event={"ID":"9b560fe3-a5b0-4a5c-895f-2565bde3f201","Type":"ContainerDied","Data":"18831d185195e9714251318baa0aafe7697e9fc8f40579d539cb3a4db7aefad7"} Jul 06 19:42:56.214630 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:56.214607 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" Jul 06 19:42:56.227317 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:56.227278 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b560fe3-a5b0-4a5c-895f-2565bde3f201-openshift-service-ca\") pod \"9b560fe3-a5b0-4a5c-895f-2565bde3f201\" (UID: \"9b560fe3-a5b0-4a5c-895f-2565bde3f201\") " Jul 06 19:42:56.227515 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:56.227330 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phvw8\" (UniqueName: \"kubernetes.io/projected/9b560fe3-a5b0-4a5c-895f-2565bde3f201-kube-api-access-phvw8\") pod \"9b560fe3-a5b0-4a5c-895f-2565bde3f201\" (UID: \"9b560fe3-a5b0-4a5c-895f-2565bde3f201\") " Jul 06 19:42:56.227515 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:56.227359 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b560fe3-a5b0-4a5c-895f-2565bde3f201-maas-api-tls\") pod \"9b560fe3-a5b0-4a5c-895f-2565bde3f201\" (UID: \"9b560fe3-a5b0-4a5c-895f-2565bde3f201\") " Jul 06 19:42:56.227775 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:56.227744 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b560fe3-a5b0-4a5c-895f-2565bde3f201-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9b560fe3-a5b0-4a5c-895f-2565bde3f201" (UID: "9b560fe3-a5b0-4a5c-895f-2565bde3f201"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:42:56.229534 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:56.229501 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b560fe3-a5b0-4a5c-895f-2565bde3f201-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9b560fe3-a5b0-4a5c-895f-2565bde3f201" (UID: "9b560fe3-a5b0-4a5c-895f-2565bde3f201"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:42:56.229636 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:56.229575 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b560fe3-a5b0-4a5c-895f-2565bde3f201-kube-api-access-phvw8" (OuterVolumeSpecName: "kube-api-access-phvw8") pod "9b560fe3-a5b0-4a5c-895f-2565bde3f201" (UID: "9b560fe3-a5b0-4a5c-895f-2565bde3f201"). InnerVolumeSpecName "kube-api-access-phvw8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:42:56.328369 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:56.328278 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b560fe3-a5b0-4a5c-895f-2565bde3f201-openshift-service-ca\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:42:56.328369 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:56.328307 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phvw8\" (UniqueName: \"kubernetes.io/projected/9b560fe3-a5b0-4a5c-895f-2565bde3f201-kube-api-access-phvw8\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:42:56.328369 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:56.328317 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b560fe3-a5b0-4a5c-895f-2565bde3f201-maas-api-tls\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:42:57.160943 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:57.160916 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" Jul 06 19:42:57.161370 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:57.160944 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq" event={"ID":"9b560fe3-a5b0-4a5c-895f-2565bde3f201","Type":"ContainerDied","Data":"ba92393cf4f46f91d3c671a3cc381514e1cb9eea00c81b146ff76812e81f13da"} Jul 06 19:42:57.161370 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:57.160998 2557 scope.go:117] "RemoveContainer" containerID="18831d185195e9714251318baa0aafe7697e9fc8f40579d539cb3a4db7aefad7" Jul 06 19:42:57.183257 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:57.183227 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq"] Jul 06 19:42:57.188328 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:57.188299 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-27c473-8654549995-nxwzq"] Jul 06 19:42:58.196537 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:42:58.196503 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b560fe3-a5b0-4a5c-895f-2565bde3f201" path="/var/lib/kubelet/pods/9b560fe3-a5b0-4a5c-895f-2565bde3f201/volumes" Jul 06 19:43:01.826795 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:01.826756 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:01.920393 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:01.920358 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:07.499533 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:07.499503 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:09.479305 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:09.479273 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:10.629691 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:10.629656 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:11.610321 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:11.610289 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:11.822577 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:11.822543 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:12.020401 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:12.020366 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:12.149274 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:12.149224 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:12.360181 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:12.360073 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:12.529077 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:12.529044 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:12.864963 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:12.864932 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x"] Jul 06 19:43:12.865540 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:12.865492 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" podUID="6c81b264-30e9-4ddb-b483-eb29e6640b0f" containerName="maas-api" containerID="cri-o://3e5263b58af6f028c6e2d7d24733bd09229ebbcabbd115821a24424e9edba760" gracePeriod=30 Jul 06 19:43:12.902021 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:12.901988 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:13.003071 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.003042 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:13.121677 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.121609 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" Jul 06 19:43:13.187820 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.187788 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c81b264-30e9-4ddb-b483-eb29e6640b0f-openshift-service-ca\") pod \"6c81b264-30e9-4ddb-b483-eb29e6640b0f\" (UID: \"6c81b264-30e9-4ddb-b483-eb29e6640b0f\") " Jul 06 19:43:13.187993 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.187845 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6rb5\" (UniqueName: \"kubernetes.io/projected/6c81b264-30e9-4ddb-b483-eb29e6640b0f-kube-api-access-q6rb5\") pod \"6c81b264-30e9-4ddb-b483-eb29e6640b0f\" (UID: \"6c81b264-30e9-4ddb-b483-eb29e6640b0f\") " Jul 06 19:43:13.187993 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.187942 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c81b264-30e9-4ddb-b483-eb29e6640b0f-maas-api-tls\") pod \"6c81b264-30e9-4ddb-b483-eb29e6640b0f\" (UID: \"6c81b264-30e9-4ddb-b483-eb29e6640b0f\") " Jul 06 19:43:13.188242 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.188217 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c81b264-30e9-4ddb-b483-eb29e6640b0f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6c81b264-30e9-4ddb-b483-eb29e6640b0f" (UID: "6c81b264-30e9-4ddb-b483-eb29e6640b0f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:43:13.189965 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.189933 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c81b264-30e9-4ddb-b483-eb29e6640b0f-kube-api-access-q6rb5" (OuterVolumeSpecName: "kube-api-access-q6rb5") pod "6c81b264-30e9-4ddb-b483-eb29e6640b0f" (UID: "6c81b264-30e9-4ddb-b483-eb29e6640b0f"). InnerVolumeSpecName "kube-api-access-q6rb5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:43:13.190090 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.189987 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c81b264-30e9-4ddb-b483-eb29e6640b0f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6c81b264-30e9-4ddb-b483-eb29e6640b0f" (UID: "6c81b264-30e9-4ddb-b483-eb29e6640b0f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:43:13.238457 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.238398 2557 generic.go:356] "Generic (PLEG): container finished" podID="6c81b264-30e9-4ddb-b483-eb29e6640b0f" containerID="3e5263b58af6f028c6e2d7d24733bd09229ebbcabbd115821a24424e9edba760" exitCode=0 Jul 06 19:43:13.238621 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.238471 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" Jul 06 19:43:13.238621 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.238493 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" event={"ID":"6c81b264-30e9-4ddb-b483-eb29e6640b0f","Type":"ContainerDied","Data":"3e5263b58af6f028c6e2d7d24733bd09229ebbcabbd115821a24424e9edba760"} Jul 06 19:43:13.238621 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.238553 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x" event={"ID":"6c81b264-30e9-4ddb-b483-eb29e6640b0f","Type":"ContainerDied","Data":"c65a9a1594fb0430cc95ee618d85465e413146adc5e158216ce7d7b93eb1f72b"} Jul 06 19:43:13.238621 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.238584 2557 scope.go:117] "RemoveContainer" containerID="3e5263b58af6f028c6e2d7d24733bd09229ebbcabbd115821a24424e9edba760" Jul 06 19:43:13.248105 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.248087 2557 scope.go:117] "RemoveContainer" containerID="3e5263b58af6f028c6e2d7d24733bd09229ebbcabbd115821a24424e9edba760" Jul 06 19:43:13.248413 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:43:13.248394 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e5263b58af6f028c6e2d7d24733bd09229ebbcabbd115821a24424e9edba760\": container with ID starting with 3e5263b58af6f028c6e2d7d24733bd09229ebbcabbd115821a24424e9edba760 not found: ID does not exist" containerID="3e5263b58af6f028c6e2d7d24733bd09229ebbcabbd115821a24424e9edba760" Jul 06 19:43:13.248476 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.248423 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e5263b58af6f028c6e2d7d24733bd09229ebbcabbd115821a24424e9edba760"} err="failed to get container status \"3e5263b58af6f028c6e2d7d24733bd09229ebbcabbd115821a24424e9edba760\": rpc error: code = NotFound desc = could not find container \"3e5263b58af6f028c6e2d7d24733bd09229ebbcabbd115821a24424e9edba760\": container with ID starting with 3e5263b58af6f028c6e2d7d24733bd09229ebbcabbd115821a24424e9edba760 not found: ID does not exist" Jul 06 19:43:13.262253 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.262227 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x"] Jul 06 19:43:13.269250 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.269224 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-dd3bb9-76974d548d-l9w7x"] Jul 06 19:43:13.289706 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.289679 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c81b264-30e9-4ddb-b483-eb29e6640b0f-maas-api-tls\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:43:13.289706 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.289706 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c81b264-30e9-4ddb-b483-eb29e6640b0f-openshift-service-ca\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:43:13.289815 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.289717 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6rb5\" (UniqueName: \"kubernetes.io/projected/6c81b264-30e9-4ddb-b483-eb29e6640b0f-kube-api-access-q6rb5\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:43:13.335647 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.335594 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:13.435591 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.435519 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:13.514862 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:13.514832 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:14.196931 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:14.196900 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c81b264-30e9-4ddb-b483-eb29e6640b0f" path="/var/lib/kubelet/pods/6c81b264-30e9-4ddb-b483-eb29e6640b0f/volumes" Jul 06 19:43:20.400126 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:20.400099 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:20.551001 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:20.550973 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:20.775413 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:20.775385 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:21.158348 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:21.158260 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:21.596066 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:21.596030 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:22.089310 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:22.089280 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:24.747536 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:24.747490 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:24.856386 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:24.856354 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:24.915672 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:24.915641 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z"] Jul 06 19:43:24.916090 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:24.916073 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c81b264-30e9-4ddb-b483-eb29e6640b0f" containerName="maas-api" Jul 06 19:43:24.916192 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:24.916094 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c81b264-30e9-4ddb-b483-eb29e6640b0f" containerName="maas-api" Jul 06 19:43:24.916192 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:24.916120 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b560fe3-a5b0-4a5c-895f-2565bde3f201" containerName="maas-api" Jul 06 19:43:24.916192 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:24.916128 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b560fe3-a5b0-4a5c-895f-2565bde3f201" containerName="maas-api" Jul 06 19:43:24.916343 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:24.916253 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b560fe3-a5b0-4a5c-895f-2565bde3f201" containerName="maas-api" Jul 06 19:43:24.916343 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:24.916267 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c81b264-30e9-4ddb-b483-eb29e6640b0f" containerName="maas-api" Jul 06 19:43:24.920855 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:24.920833 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" Jul 06 19:43:24.925681 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:24.925661 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-5c92ad-serving-cert\"" Jul 06 19:43:24.925814 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:24.925698 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-zxgdj\"" Jul 06 19:43:24.937713 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:24.937683 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z"] Jul 06 19:43:24.970167 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:24.970137 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:25.099607 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:25.099523 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z\" (UID: \"e0aa6de9-1f91-4cf1-96b3-b42b56388d75\") " pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" Jul 06 19:43:25.099607 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:25.099570 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-maas-api-tls\") pod \"maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z\" (UID: \"e0aa6de9-1f91-4cf1-96b3-b42b56388d75\") " pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" Jul 06 19:43:25.099801 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:25.099615 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b27ft\" (UniqueName: \"kubernetes.io/projected/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-kube-api-access-b27ft\") pod \"maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z\" (UID: \"e0aa6de9-1f91-4cf1-96b3-b42b56388d75\") " pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" Jul 06 19:43:25.200688 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:25.200650 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z\" (UID: \"e0aa6de9-1f91-4cf1-96b3-b42b56388d75\") " pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" Jul 06 19:43:25.200893 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:25.200703 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-maas-api-tls\") pod \"maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z\" (UID: \"e0aa6de9-1f91-4cf1-96b3-b42b56388d75\") " pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" Jul 06 19:43:25.200893 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:25.200741 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b27ft\" (UniqueName: \"kubernetes.io/projected/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-kube-api-access-b27ft\") pod \"maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z\" (UID: \"e0aa6de9-1f91-4cf1-96b3-b42b56388d75\") " pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" Jul 06 19:43:25.201360 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:25.201334 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z\" (UID: \"e0aa6de9-1f91-4cf1-96b3-b42b56388d75\") " pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" Jul 06 19:43:25.203169 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:25.203146 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-maas-api-tls\") pod \"maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z\" (UID: \"e0aa6de9-1f91-4cf1-96b3-b42b56388d75\") " pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" Jul 06 19:43:25.223826 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:25.223795 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b27ft\" (UniqueName: \"kubernetes.io/projected/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-kube-api-access-b27ft\") pod \"maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z\" (UID: \"e0aa6de9-1f91-4cf1-96b3-b42b56388d75\") " pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" Jul 06 19:43:25.230703 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:25.230662 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" Jul 06 19:43:25.288569 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:25.288430 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:25.386410 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:25.386385 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z"] Jul 06 19:43:25.388837 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:43:25.388809 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0aa6de9_1f91_4cf1_96b3_b42b56388d75.slice/crio-3dc932e13d820a4215a41780911eaed181dd3a6486d1defd6ff1f3809a53a1aa WatchSource:0}: Error finding container 3dc932e13d820a4215a41780911eaed181dd3a6486d1defd6ff1f3809a53a1aa: Status 404 returned error can't find the container with id 3dc932e13d820a4215a41780911eaed181dd3a6486d1defd6ff1f3809a53a1aa Jul 06 19:43:25.615293 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:25.615266 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:25.713739 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:25.713652 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:26.305033 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:26.305004 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" event={"ID":"e0aa6de9-1f91-4cf1-96b3-b42b56388d75","Type":"ContainerStarted","Data":"a81dd3c276ebe5848d6e085f57112d03316bf86ff52b8fbe44f86f8b7cd2c0f4"} Jul 06 19:43:26.305408 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:26.305053 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" event={"ID":"e0aa6de9-1f91-4cf1-96b3-b42b56388d75","Type":"ContainerStarted","Data":"3dc932e13d820a4215a41780911eaed181dd3a6486d1defd6ff1f3809a53a1aa"} Jul 06 19:43:26.305408 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:26.305143 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" Jul 06 19:43:26.338720 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:26.338673 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" podStartSLOduration=1.953312152 podStartE2EDuration="2.338656786s" totalImagesPullingTime="385.344634ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:43:26.33770644 +0000 UTC m=+2646.698870233" watchObservedRunningTime="2026-07-06 19:43:26.338656786 +0000 UTC m=+2646.699820578" Jul 06 19:43:29.269703 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:29.269663 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:29.595717 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:29.595639 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:29.944168 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:29.944076 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:30.047380 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:30.047351 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:32.315741 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:32.315708 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" Jul 06 19:43:32.838634 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:32.838595 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:32.944855 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:32.944814 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm"] Jul 06 19:43:32.948334 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:32.948310 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" Jul 06 19:43:32.954213 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:32.954193 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-5f29b8-serving-cert\"" Jul 06 19:43:32.966477 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:32.966456 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm"] Jul 06 19:43:32.975856 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:32.975833 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0127ae5a-20d6-4223-9deb-655719968dd4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm\" (UID: \"0127ae5a-20d6-4223-9deb-655719968dd4\") " pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" Jul 06 19:43:32.975973 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:32.975934 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0127ae5a-20d6-4223-9deb-655719968dd4-maas-api-tls\") pod \"maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm\" (UID: \"0127ae5a-20d6-4223-9deb-655719968dd4\") " pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" Jul 06 19:43:32.976044 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:32.975977 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rwvg\" (UniqueName: \"kubernetes.io/projected/0127ae5a-20d6-4223-9deb-655719968dd4-kube-api-access-9rwvg\") pod \"maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm\" (UID: \"0127ae5a-20d6-4223-9deb-655719968dd4\") " pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" Jul 06 19:43:33.077034 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:33.076998 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0127ae5a-20d6-4223-9deb-655719968dd4-maas-api-tls\") pod \"maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm\" (UID: \"0127ae5a-20d6-4223-9deb-655719968dd4\") " pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" Jul 06 19:43:33.077211 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:33.077049 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rwvg\" (UniqueName: \"kubernetes.io/projected/0127ae5a-20d6-4223-9deb-655719968dd4-kube-api-access-9rwvg\") pod \"maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm\" (UID: \"0127ae5a-20d6-4223-9deb-655719968dd4\") " pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" Jul 06 19:43:33.077211 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:33.077089 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0127ae5a-20d6-4223-9deb-655719968dd4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm\" (UID: \"0127ae5a-20d6-4223-9deb-655719968dd4\") " pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" Jul 06 19:43:33.077770 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:33.077744 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0127ae5a-20d6-4223-9deb-655719968dd4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm\" (UID: \"0127ae5a-20d6-4223-9deb-655719968dd4\") " pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" Jul 06 19:43:33.079428 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:33.079402 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0127ae5a-20d6-4223-9deb-655719968dd4-maas-api-tls\") pod \"maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm\" (UID: \"0127ae5a-20d6-4223-9deb-655719968dd4\") " pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" Jul 06 19:43:33.096533 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:33.096466 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rwvg\" (UniqueName: \"kubernetes.io/projected/0127ae5a-20d6-4223-9deb-655719968dd4-kube-api-access-9rwvg\") pod \"maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm\" (UID: \"0127ae5a-20d6-4223-9deb-655719968dd4\") " pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" Jul 06 19:43:33.110188 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:33.110160 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:33.259950 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:33.259911 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" Jul 06 19:43:33.303628 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:33.303565 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:33.403826 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:33.403800 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm"] Jul 06 19:43:33.405943 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:43:33.405913 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0127ae5a_20d6_4223_9deb_655719968dd4.slice/crio-fd14d0599743212715556a27d2915f3ce8a110602709831d5bb23842751c5acb WatchSource:0}: Error finding container fd14d0599743212715556a27d2915f3ce8a110602709831d5bb23842751c5acb: Status 404 returned error can't find the container with id fd14d0599743212715556a27d2915f3ce8a110602709831d5bb23842751c5acb Jul 06 19:43:33.711300 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:33.711213 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:33.871727 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:33.871692 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:34.343643 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:34.343606 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" event={"ID":"0127ae5a-20d6-4223-9deb-655719968dd4","Type":"ContainerStarted","Data":"3d17cbc264bfe836bb4ea933eb1259e6c29d8c852333844b90f11de4d905040c"} Jul 06 19:43:34.343843 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:34.343654 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" event={"ID":"0127ae5a-20d6-4223-9deb-655719968dd4","Type":"ContainerStarted","Data":"fd14d0599743212715556a27d2915f3ce8a110602709831d5bb23842751c5acb"} Jul 06 19:43:34.343843 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:34.343735 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" Jul 06 19:43:34.375346 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:34.375291 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" podStartSLOduration=2.008347095 podStartE2EDuration="2.375274075s" totalImagesPullingTime="366.92698ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:43:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:43:34.374933945 +0000 UTC m=+2654.736097738" watchObservedRunningTime="2026-07-06 19:43:34.375274075 +0000 UTC m=+2654.736437868" Jul 06 19:43:37.311452 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:37.311401 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:40.353667 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:40.353633 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" Jul 06 19:43:47.535986 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:47.535955 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:47.740569 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:47.740539 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:54.647603 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:54.647567 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:54.743601 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:54.743565 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:56.027158 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:56.027122 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:56.176211 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:43:56.176178 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:44:06.651827 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:44:06.651779 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:44:32.548213 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:44:32.548062 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:44:32.649525 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:44:32.649491 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:44:34.386869 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:44:34.386833 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:44:43.887784 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:44:43.887724 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:45:22.952624 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:45:22.952585 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:45:59.561480 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:45:59.561423 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:46:46.204755 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:46:46.204713 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:46:47.191955 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:46:47.191914 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:47:07.138132 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:47:07.138094 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:47:31.736296 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:47:31.736168 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:47:31.801924 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:47:31.801896 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:47:34.069427 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:47:34.069397 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:22.915095 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:48:22.915049 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:24.394783 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:48:24.394748 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:40.543038 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:48:40.542999 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:48.520889 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:48:48.520794 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:48.621117 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:48:48.621078 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:56.911253 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:48:56.911221 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:57.010496 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:48:57.010419 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:28.307922 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:49:28.307882 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:33.251245 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:49:33.251197 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:33.351058 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:49:33.351005 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:51.091219 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:49:51.091183 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:51.178142 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:49:51.178092 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:56.083380 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:49:56.083342 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:56.148431 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:49:56.148400 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:58.352430 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:49:58.352396 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:59.835602 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:49:59.835541 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:59.935721 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:49:59.935686 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:00.006791 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:50:00.006749 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:22.186927 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:50:22.186799 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:23.106654 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:50:23.106622 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:28.851834 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:50:28.851799 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:54.524030 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:50:54.523990 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:54.619750 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:50:54.619721 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:55.083067 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:50:55.083004 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:55.251871 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:50:55.251837 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:55.454168 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:50:55.454075 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:50:55.606781 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:50:55.606728 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:17.169512 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.169464 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z"] Jul 06 19:51:17.170040 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.169752 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" podUID="e0aa6de9-1f91-4cf1-96b3-b42b56388d75" containerName="maas-api" containerID="cri-o://a81dd3c276ebe5848d6e085f57112d03316bf86ff52b8fbe44f86f8b7cd2c0f4" gracePeriod=30 Jul 06 19:51:17.311070 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.311030 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" podUID="e0aa6de9-1f91-4cf1-96b3-b42b56388d75" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.44:8443/health\": dial tcp 10.134.0.44:8443: connect: connection refused" Jul 06 19:51:17.417168 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.417143 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" Jul 06 19:51:17.437706 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.437622 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b27ft\" (UniqueName: \"kubernetes.io/projected/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-kube-api-access-b27ft\") pod \"e0aa6de9-1f91-4cf1-96b3-b42b56388d75\" (UID: \"e0aa6de9-1f91-4cf1-96b3-b42b56388d75\") " Jul 06 19:51:17.437706 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.437696 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-openshift-service-ca\") pod \"e0aa6de9-1f91-4cf1-96b3-b42b56388d75\" (UID: \"e0aa6de9-1f91-4cf1-96b3-b42b56388d75\") " Jul 06 19:51:17.437929 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.437761 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-maas-api-tls\") pod \"e0aa6de9-1f91-4cf1-96b3-b42b56388d75\" (UID: \"e0aa6de9-1f91-4cf1-96b3-b42b56388d75\") " Jul 06 19:51:17.438074 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.438048 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e0aa6de9-1f91-4cf1-96b3-b42b56388d75" (UID: "e0aa6de9-1f91-4cf1-96b3-b42b56388d75"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:51:17.439781 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.439760 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-kube-api-access-b27ft" (OuterVolumeSpecName: "kube-api-access-b27ft") pod "e0aa6de9-1f91-4cf1-96b3-b42b56388d75" (UID: "e0aa6de9-1f91-4cf1-96b3-b42b56388d75"). InnerVolumeSpecName "kube-api-access-b27ft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:51:17.439983 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.439962 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e0aa6de9-1f91-4cf1-96b3-b42b56388d75" (UID: "e0aa6de9-1f91-4cf1-96b3-b42b56388d75"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:17.538453 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.538406 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-openshift-service-ca\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:51:17.538628 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.538465 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-maas-api-tls\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:51:17.538628 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.538477 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b27ft\" (UniqueName: \"kubernetes.io/projected/e0aa6de9-1f91-4cf1-96b3-b42b56388d75-kube-api-access-b27ft\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:51:17.646223 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.646186 2557 generic.go:356] "Generic (PLEG): container finished" podID="e0aa6de9-1f91-4cf1-96b3-b42b56388d75" containerID="a81dd3c276ebe5848d6e085f57112d03316bf86ff52b8fbe44f86f8b7cd2c0f4" exitCode=0 Jul 06 19:51:17.646223 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.646229 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" Jul 06 19:51:17.646526 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.646277 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" event={"ID":"e0aa6de9-1f91-4cf1-96b3-b42b56388d75","Type":"ContainerDied","Data":"a81dd3c276ebe5848d6e085f57112d03316bf86ff52b8fbe44f86f8b7cd2c0f4"} Jul 06 19:51:17.646526 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.646339 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z" event={"ID":"e0aa6de9-1f91-4cf1-96b3-b42b56388d75","Type":"ContainerDied","Data":"3dc932e13d820a4215a41780911eaed181dd3a6486d1defd6ff1f3809a53a1aa"} Jul 06 19:51:17.646526 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.646375 2557 scope.go:117] "RemoveContainer" containerID="a81dd3c276ebe5848d6e085f57112d03316bf86ff52b8fbe44f86f8b7cd2c0f4" Jul 06 19:51:17.658280 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.658261 2557 scope.go:117] "RemoveContainer" containerID="a81dd3c276ebe5848d6e085f57112d03316bf86ff52b8fbe44f86f8b7cd2c0f4" Jul 06 19:51:17.658604 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:51:17.658581 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a81dd3c276ebe5848d6e085f57112d03316bf86ff52b8fbe44f86f8b7cd2c0f4\": container with ID starting with a81dd3c276ebe5848d6e085f57112d03316bf86ff52b8fbe44f86f8b7cd2c0f4 not found: ID does not exist" containerID="a81dd3c276ebe5848d6e085f57112d03316bf86ff52b8fbe44f86f8b7cd2c0f4" Jul 06 19:51:17.658704 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.658611 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a81dd3c276ebe5848d6e085f57112d03316bf86ff52b8fbe44f86f8b7cd2c0f4"} err="failed to get container status \"a81dd3c276ebe5848d6e085f57112d03316bf86ff52b8fbe44f86f8b7cd2c0f4\": rpc error: code = NotFound desc = could not find container \"a81dd3c276ebe5848d6e085f57112d03316bf86ff52b8fbe44f86f8b7cd2c0f4\": container with ID starting with a81dd3c276ebe5848d6e085f57112d03316bf86ff52b8fbe44f86f8b7cd2c0f4 not found: ID does not exist" Jul 06 19:51:17.672959 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.672930 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z"] Jul 06 19:51:17.683078 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:17.683054 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-5c92ad-58444f5d4b-5988z"] Jul 06 19:51:18.196430 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:18.196400 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0aa6de9-1f91-4cf1-96b3-b42b56388d75" path="/var/lib/kubelet/pods/e0aa6de9-1f91-4cf1-96b3-b42b56388d75/volumes" Jul 06 19:51:18.571345 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:18.571286 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:18.742511 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:18.742460 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:19.013348 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:19.013319 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:19.217315 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:19.217286 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:19.571088 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:19.571054 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:19.958920 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:19.958826 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:20.289010 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:20.288979 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:20.433085 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:20.433045 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:20.597910 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:20.597818 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm"] Jul 06 19:51:20.598087 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:20.598062 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" podUID="0127ae5a-20d6-4223-9deb-655719968dd4" containerName="maas-api" containerID="cri-o://3d17cbc264bfe836bb4ea933eb1259e6c29d8c852333844b90f11de4d905040c" gracePeriod=30 Jul 06 19:51:20.620095 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:20.620069 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:20.846492 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:20.846467 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" Jul 06 19:51:20.869778 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:20.869674 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0127ae5a-20d6-4223-9deb-655719968dd4-openshift-service-ca\") pod \"0127ae5a-20d6-4223-9deb-655719968dd4\" (UID: \"0127ae5a-20d6-4223-9deb-655719968dd4\") " Jul 06 19:51:20.869778 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:20.869770 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rwvg\" (UniqueName: \"kubernetes.io/projected/0127ae5a-20d6-4223-9deb-655719968dd4-kube-api-access-9rwvg\") pod \"0127ae5a-20d6-4223-9deb-655719968dd4\" (UID: \"0127ae5a-20d6-4223-9deb-655719968dd4\") " Jul 06 19:51:20.870017 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:20.869821 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0127ae5a-20d6-4223-9deb-655719968dd4-maas-api-tls\") pod \"0127ae5a-20d6-4223-9deb-655719968dd4\" (UID: \"0127ae5a-20d6-4223-9deb-655719968dd4\") " Jul 06 19:51:20.870071 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:20.870038 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0127ae5a-20d6-4223-9deb-655719968dd4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0127ae5a-20d6-4223-9deb-655719968dd4" (UID: "0127ae5a-20d6-4223-9deb-655719968dd4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:51:20.871899 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:20.871859 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0127ae5a-20d6-4223-9deb-655719968dd4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0127ae5a-20d6-4223-9deb-655719968dd4" (UID: "0127ae5a-20d6-4223-9deb-655719968dd4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:20.872025 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:20.872006 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0127ae5a-20d6-4223-9deb-655719968dd4-kube-api-access-9rwvg" (OuterVolumeSpecName: "kube-api-access-9rwvg") pod "0127ae5a-20d6-4223-9deb-655719968dd4" (UID: "0127ae5a-20d6-4223-9deb-655719968dd4"). InnerVolumeSpecName "kube-api-access-9rwvg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:51:20.971429 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:20.971400 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0127ae5a-20d6-4223-9deb-655719968dd4-openshift-service-ca\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:51:20.971429 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:20.971425 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rwvg\" (UniqueName: \"kubernetes.io/projected/0127ae5a-20d6-4223-9deb-655719968dd4-kube-api-access-9rwvg\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:51:20.971657 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:20.971465 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0127ae5a-20d6-4223-9deb-655719968dd4-maas-api-tls\") on node \"ip-10-0-131-230.ec2.internal\" DevicePath \"\"" Jul 06 19:51:21.669114 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:21.669079 2557 generic.go:356] "Generic (PLEG): container finished" podID="0127ae5a-20d6-4223-9deb-655719968dd4" containerID="3d17cbc264bfe836bb4ea933eb1259e6c29d8c852333844b90f11de4d905040c" exitCode=0 Jul 06 19:51:21.669547 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:21.669129 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" Jul 06 19:51:21.669547 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:21.669167 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" event={"ID":"0127ae5a-20d6-4223-9deb-655719968dd4","Type":"ContainerDied","Data":"3d17cbc264bfe836bb4ea933eb1259e6c29d8c852333844b90f11de4d905040c"} Jul 06 19:51:21.669547 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:21.669225 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm" event={"ID":"0127ae5a-20d6-4223-9deb-655719968dd4","Type":"ContainerDied","Data":"fd14d0599743212715556a27d2915f3ce8a110602709831d5bb23842751c5acb"} Jul 06 19:51:21.669547 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:21.669255 2557 scope.go:117] "RemoveContainer" containerID="3d17cbc264bfe836bb4ea933eb1259e6c29d8c852333844b90f11de4d905040c" Jul 06 19:51:21.678845 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:21.678815 2557 scope.go:117] "RemoveContainer" containerID="3d17cbc264bfe836bb4ea933eb1259e6c29d8c852333844b90f11de4d905040c" Jul 06 19:51:21.679155 ip-10-0-131-230 kubenswrapper[2557]: E0706 19:51:21.679113 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d17cbc264bfe836bb4ea933eb1259e6c29d8c852333844b90f11de4d905040c\": container with ID starting with 3d17cbc264bfe836bb4ea933eb1259e6c29d8c852333844b90f11de4d905040c not found: ID does not exist" containerID="3d17cbc264bfe836bb4ea933eb1259e6c29d8c852333844b90f11de4d905040c" Jul 06 19:51:21.679208 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:21.679165 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d17cbc264bfe836bb4ea933eb1259e6c29d8c852333844b90f11de4d905040c"} err="failed to get container status \"3d17cbc264bfe836bb4ea933eb1259e6c29d8c852333844b90f11de4d905040c\": rpc error: code = NotFound desc = could not find container \"3d17cbc264bfe836bb4ea933eb1259e6c29d8c852333844b90f11de4d905040c\": container with ID starting with 3d17cbc264bfe836bb4ea933eb1259e6c29d8c852333844b90f11de4d905040c not found: ID does not exist" Jul 06 19:51:21.710842 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:21.710804 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm"] Jul 06 19:51:21.719690 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:21.719662 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-5f29b8-7754fbc56f-sk7vm"] Jul 06 19:51:22.197878 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:22.197846 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0127ae5a-20d6-4223-9deb-655719968dd4" path="/var/lib/kubelet/pods/0127ae5a-20d6-4223-9deb-655719968dd4/volumes" Jul 06 19:51:26.946718 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:26.946684 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:27.051239 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:27.051207 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:27.295098 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:27.295057 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:27.494243 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:27.494208 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:32.960812 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:32.960777 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-787885975b-9sszn_597c5e2d-af7f-4979-9175-d0f836161efc/maas-api/0.log" Jul 06 19:51:33.943356 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:33.943328 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m_c2780119-d899-424a-895e-9ef16b4c3cab/copy-manifests/0.log" Jul 06 19:51:33.960180 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:33.960151 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m_c2780119-d899-424a-895e-9ef16b4c3cab/manager/1.log" Jul 06 19:51:34.197308 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:34.197232 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-nzrj5_0c7f699e-49b9-4132-8c15-b0046803b3ad/postgres/0.log" Jul 06 19:51:36.404502 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:36.404470 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-gd8lr_b2d540f2-9dec-4d75-842d-815689156ee0/manager/4.log" Jul 06 19:51:36.854312 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:36.854279 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5422w_1fa0868b-64da-4c83-b866-29bc1a9748ee/discovery/0.log" Jul 06 19:51:38.165238 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:38.165211 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-wnjmx_85852870-ec89-4989-a187-94520d14393e/main/0.log" Jul 06 19:51:43.226122 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:43.226039 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:43.297137 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:43.297099 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" limit=1024 Jul 06 19:51:46.196870 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:46.196837 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k9pnm_04e067b8-9d48-42c6-8cac-0254ea3a4328/global-pull-secret-syncer/0.log" Jul 06 19:51:46.466347 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:46.466265 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2lj55_d9d93395-0529-4fa6-9cd4-ddd2ba9c5c8c/konnectivity-agent/0.log" Jul 06 19:51:46.606826 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:46.606800 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-230.ec2.internal_f78ec814e87d18142aa89255a77769d0/haproxy/0.log" Jul 06 19:51:52.477363 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:52.477336 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-gd8lr_b2d540f2-9dec-4d75-842d-815689156ee0/manager/3.log" Jul 06 19:51:52.511787 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:52.511699 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-gd8lr_b2d540f2-9dec-4d75-842d-815689156ee0/manager/4.log" Jul 06 19:51:54.204781 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:54.204702 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4de5ee22-0455-4eb2-81ad-9f8e5e686b14/alertmanager/0.log" Jul 06 19:51:54.244736 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:54.244704 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4de5ee22-0455-4eb2-81ad-9f8e5e686b14/config-reloader/0.log" Jul 06 19:51:54.291855 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:54.291832 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4de5ee22-0455-4eb2-81ad-9f8e5e686b14/kube-rbac-proxy-web/0.log" Jul 06 19:51:54.328987 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:54.328963 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4de5ee22-0455-4eb2-81ad-9f8e5e686b14/kube-rbac-proxy/0.log" Jul 06 19:51:54.372357 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:54.372331 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4de5ee22-0455-4eb2-81ad-9f8e5e686b14/kube-rbac-proxy-metric/0.log" Jul 06 19:51:54.413273 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:54.413241 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4de5ee22-0455-4eb2-81ad-9f8e5e686b14/prom-label-proxy/0.log" Jul 06 19:51:54.441370 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:54.441347 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4de5ee22-0455-4eb2-81ad-9f8e5e686b14/init-config-reloader/0.log" Jul 06 19:51:54.528168 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:54.528139 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-gsfgd_48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153/kube-state-metrics/0.log" Jul 06 19:51:54.569297 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:54.569248 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-gsfgd_48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153/kube-rbac-proxy-main/0.log" Jul 06 19:51:54.598349 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:54.598328 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-gsfgd_48fb1c5a-c99e-4dc4-88a3-f6b32aa8d153/kube-rbac-proxy-self/0.log" Jul 06 19:51:54.629001 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:54.628974 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-57b7fccbbf-5c2nh_03b694d1-fffd-4360-867b-72db963a7a15/metrics-server/0.log" Jul 06 19:51:54.666366 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:54.666333 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-d84pd_5c8854d1-590f-4b85-8925-1cd1eb42dc13/monitoring-plugin/0.log" Jul 06 19:51:54.962944 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:54.962861 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vnsbs_5e3dcc7d-c527-43c9-8d48-35fccf277853/node-exporter/0.log" Jul 06 19:51:54.996374 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:54.996344 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vnsbs_5e3dcc7d-c527-43c9-8d48-35fccf277853/kube-rbac-proxy/0.log" Jul 06 19:51:55.024296 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.024266 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vnsbs_5e3dcc7d-c527-43c9-8d48-35fccf277853/init-textfile/0.log" Jul 06 19:51:55.061257 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.061228 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-9j6j7_77d08ee5-6984-47b3-8e6c-9645bc5afe52/kube-rbac-proxy-main/0.log" Jul 06 19:51:55.106940 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.106918 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-9j6j7_77d08ee5-6984-47b3-8e6c-9645bc5afe52/kube-rbac-proxy-self/0.log" Jul 06 19:51:55.139662 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.139634 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-9j6j7_77d08ee5-6984-47b3-8e6c-9645bc5afe52/openshift-state-metrics/0.log" Jul 06 19:51:55.184832 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.184806 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_78fdb525-75bb-444c-a0c2-435dcac110e8/prometheus/0.log" Jul 06 19:51:55.221056 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.220974 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_78fdb525-75bb-444c-a0c2-435dcac110e8/config-reloader/0.log" Jul 06 19:51:55.254076 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.254053 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_78fdb525-75bb-444c-a0c2-435dcac110e8/thanos-sidecar/0.log" Jul 06 19:51:55.301119 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.301091 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_78fdb525-75bb-444c-a0c2-435dcac110e8/kube-rbac-proxy-web/0.log" Jul 06 19:51:55.325911 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.325890 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_78fdb525-75bb-444c-a0c2-435dcac110e8/kube-rbac-proxy/0.log" Jul 06 19:51:55.359216 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.359183 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_78fdb525-75bb-444c-a0c2-435dcac110e8/kube-rbac-proxy-thanos/0.log" Jul 06 19:51:55.439276 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.439245 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_78fdb525-75bb-444c-a0c2-435dcac110e8/init-config-reloader/0.log" Jul 06 19:51:55.483648 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.483573 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-fcj9t_2d6e5eba-0788-469a-858a-fdd5cf5be8a8/prometheus-operator/0.log" Jul 06 19:51:55.519819 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.519794 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-fcj9t_2d6e5eba-0788-469a-858a-fdd5cf5be8a8/kube-rbac-proxy/0.log" Jul 06 19:51:55.552193 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.552171 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-bkrg4_76656ba1-38dc-4c56-8667-2624920898ed/prometheus-operator-admission-webhook/0.log" Jul 06 19:51:55.593936 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.593908 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6455cb7df7-swbff_8c47284b-8397-4f76-9cb9-b1ce93b25e8f/telemeter-client/0.log" Jul 06 19:51:55.620770 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.620748 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6455cb7df7-swbff_8c47284b-8397-4f76-9cb9-b1ce93b25e8f/reload/0.log" Jul 06 19:51:55.647510 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.647476 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6455cb7df7-swbff_8c47284b-8397-4f76-9cb9-b1ce93b25e8f/kube-rbac-proxy/0.log" Jul 06 19:51:55.682871 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.682839 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b85fc5cc7-wpfrp_5abc161e-79ec-4ba0-b063-084f93dfe838/thanos-query/0.log" Jul 06 19:51:55.716423 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.716395 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b85fc5cc7-wpfrp_5abc161e-79ec-4ba0-b063-084f93dfe838/kube-rbac-proxy-web/0.log" Jul 06 19:51:55.741161 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.741134 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b85fc5cc7-wpfrp_5abc161e-79ec-4ba0-b063-084f93dfe838/kube-rbac-proxy/0.log" Jul 06 19:51:55.766262 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.766237 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b85fc5cc7-wpfrp_5abc161e-79ec-4ba0-b063-084f93dfe838/prom-label-proxy/0.log" Jul 06 19:51:55.792294 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.792271 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b85fc5cc7-wpfrp_5abc161e-79ec-4ba0-b063-084f93dfe838/kube-rbac-proxy-rules/0.log" Jul 06 19:51:55.818792 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:55.818767 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b85fc5cc7-wpfrp_5abc161e-79ec-4ba0-b063-084f93dfe838/kube-rbac-proxy-metrics/0.log" Jul 06 19:51:56.203683 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.203656 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz"] Jul 06 19:51:56.204038 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.204027 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0aa6de9-1f91-4cf1-96b3-b42b56388d75" containerName="maas-api" Jul 06 19:51:56.204092 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.204040 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0aa6de9-1f91-4cf1-96b3-b42b56388d75" containerName="maas-api" Jul 06 19:51:56.204092 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.204064 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0127ae5a-20d6-4223-9deb-655719968dd4" containerName="maas-api" Jul 06 19:51:56.204092 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.204070 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0127ae5a-20d6-4223-9deb-655719968dd4" containerName="maas-api" Jul 06 19:51:56.204190 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.204122 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0aa6de9-1f91-4cf1-96b3-b42b56388d75" containerName="maas-api" Jul 06 19:51:56.204190 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.204133 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0127ae5a-20d6-4223-9deb-655719968dd4" containerName="maas-api" Jul 06 19:51:56.207216 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.207201 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.210947 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.210928 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nqdz2\"/\"openshift-service-ca.crt\"" Jul 06 19:51:56.211072 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.210932 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nqdz2\"/\"kube-root-ca.crt\"" Jul 06 19:51:56.211564 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.211546 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nqdz2\"/\"default-dockercfg-6xq4g\"" Jul 06 19:51:56.218751 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.218730 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz"] Jul 06 19:51:56.302848 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.302805 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/49ecd2e6-5058-472d-9590-888af0ceb626-podres\") pod \"perf-node-gather-daemonset-nh8lz\" (UID: \"49ecd2e6-5058-472d-9590-888af0ceb626\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.303353 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.302884 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/49ecd2e6-5058-472d-9590-888af0ceb626-lib-modules\") pod \"perf-node-gather-daemonset-nh8lz\" (UID: \"49ecd2e6-5058-472d-9590-888af0ceb626\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.303353 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.302916 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/49ecd2e6-5058-472d-9590-888af0ceb626-sys\") pod \"perf-node-gather-daemonset-nh8lz\" (UID: \"49ecd2e6-5058-472d-9590-888af0ceb626\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.303353 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.302946 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/49ecd2e6-5058-472d-9590-888af0ceb626-proc\") pod \"perf-node-gather-daemonset-nh8lz\" (UID: \"49ecd2e6-5058-472d-9590-888af0ceb626\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.303353 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.302996 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4ndx\" (UniqueName: \"kubernetes.io/projected/49ecd2e6-5058-472d-9590-888af0ceb626-kube-api-access-p4ndx\") pod \"perf-node-gather-daemonset-nh8lz\" (UID: \"49ecd2e6-5058-472d-9590-888af0ceb626\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.404662 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.404578 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/49ecd2e6-5058-472d-9590-888af0ceb626-podres\") pod \"perf-node-gather-daemonset-nh8lz\" (UID: \"49ecd2e6-5058-472d-9590-888af0ceb626\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.404662 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.404619 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/49ecd2e6-5058-472d-9590-888af0ceb626-lib-modules\") pod \"perf-node-gather-daemonset-nh8lz\" (UID: \"49ecd2e6-5058-472d-9590-888af0ceb626\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.404662 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.404638 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/49ecd2e6-5058-472d-9590-888af0ceb626-sys\") pod \"perf-node-gather-daemonset-nh8lz\" (UID: \"49ecd2e6-5058-472d-9590-888af0ceb626\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.404916 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.404670 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/49ecd2e6-5058-472d-9590-888af0ceb626-proc\") pod \"perf-node-gather-daemonset-nh8lz\" (UID: \"49ecd2e6-5058-472d-9590-888af0ceb626\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.404916 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.404703 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4ndx\" (UniqueName: \"kubernetes.io/projected/49ecd2e6-5058-472d-9590-888af0ceb626-kube-api-access-p4ndx\") pod \"perf-node-gather-daemonset-nh8lz\" (UID: \"49ecd2e6-5058-472d-9590-888af0ceb626\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.404916 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.404734 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/49ecd2e6-5058-472d-9590-888af0ceb626-sys\") pod \"perf-node-gather-daemonset-nh8lz\" (UID: \"49ecd2e6-5058-472d-9590-888af0ceb626\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.404916 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.404748 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/49ecd2e6-5058-472d-9590-888af0ceb626-podres\") pod \"perf-node-gather-daemonset-nh8lz\" (UID: \"49ecd2e6-5058-472d-9590-888af0ceb626\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.404916 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.404773 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/49ecd2e6-5058-472d-9590-888af0ceb626-lib-modules\") pod \"perf-node-gather-daemonset-nh8lz\" (UID: \"49ecd2e6-5058-472d-9590-888af0ceb626\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.404916 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.404811 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/49ecd2e6-5058-472d-9590-888af0ceb626-proc\") pod \"perf-node-gather-daemonset-nh8lz\" (UID: \"49ecd2e6-5058-472d-9590-888af0ceb626\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.413933 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.413908 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4ndx\" (UniqueName: \"kubernetes.io/projected/49ecd2e6-5058-472d-9590-888af0ceb626-kube-api-access-p4ndx\") pod \"perf-node-gather-daemonset-nh8lz\" (UID: \"49ecd2e6-5058-472d-9590-888af0ceb626\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.519303 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.519271 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.663521 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.663492 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz"] Jul 06 19:51:56.666070 ip-10-0-131-230 kubenswrapper[2557]: W0706 19:51:56.666039 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod49ecd2e6_5058_472d_9590_888af0ceb626.slice/crio-08681ad0b242497200da7878212ece93398b7f97d963fc96e08b52e9999e7fd7 WatchSource:0}: Error finding container 08681ad0b242497200da7878212ece93398b7f97d963fc96e08b52e9999e7fd7: Status 404 returned error can't find the container with id 08681ad0b242497200da7878212ece93398b7f97d963fc96e08b52e9999e7fd7 Jul 06 19:51:56.851721 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.851684 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" event={"ID":"49ecd2e6-5058-472d-9590-888af0ceb626","Type":"ContainerStarted","Data":"d527e7ddef63fa5973e958144ac45a9af8c638f9d05513a96bc55f6898d2b85f"} Jul 06 19:51:56.851928 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.851736 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" event={"ID":"49ecd2e6-5058-472d-9590-888af0ceb626","Type":"ContainerStarted","Data":"08681ad0b242497200da7878212ece93398b7f97d963fc96e08b52e9999e7fd7"} Jul 06 19:51:56.851928 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.851765 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:51:56.872587 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:56.872528 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" podStartSLOduration=0.872511868 podStartE2EDuration="872.511868ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:51:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:51:56.872397972 +0000 UTC m=+3157.233561764" watchObservedRunningTime="2026-07-06 19:51:56.872511868 +0000 UTC m=+3157.233675664" Jul 06 19:51:58.144039 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:51:58.144004 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-xd58h_bf18ee49-d386-429b-b848-ed10ff8f5280/download-server/0.log" Jul 06 19:52:00.199919 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:00.199890 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ptnmn_3532d5a1-91d6-426c-8e98-a6afaaec3c8e/dns/0.log" Jul 06 19:52:00.225312 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:00.225284 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ptnmn_3532d5a1-91d6-426c-8e98-a6afaaec3c8e/kube-rbac-proxy/0.log" Jul 06 19:52:00.252620 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:00.252592 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n8wgx_777e6e47-55fc-4945-81c1-c3e687f0652b/dns-node-resolver/0.log" Jul 06 19:52:00.995828 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:00.995799 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xn6qm_b9d924cf-a6f6-47d0-adae-989ebcab4a23/node-ca/0.log" Jul 06 19:52:02.185246 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:02.185210 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5422w_1fa0868b-64da-4c83-b866-29bc1a9748ee/discovery/0.log" Jul 06 19:52:02.866365 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:02.866337 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-nh8lz" Jul 06 19:52:03.410648 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:03.410620 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8sq64_b7605339-9208-4293-b7f5-0573dd8d65de/serve-healthcheck-canary/0.log" Jul 06 19:52:03.986164 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:03.986136 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q725j_dd8fd2e7-6baa-43ec-85c7-457e042c3834/kube-rbac-proxy/0.log" Jul 06 19:52:04.010246 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:04.010213 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q725j_dd8fd2e7-6baa-43ec-85c7-457e042c3834/exporter/0.log" Jul 06 19:52:04.033231 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:04.033206 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q725j_dd8fd2e7-6baa-43ec-85c7-457e042c3834/extractor/0.log" Jul 06 19:52:06.286119 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:06.286090 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5422w_1fa0868b-64da-4c83-b866-29bc1a9748ee/discovery/0.log" Jul 06 19:52:07.079109 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:07.079082 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-787885975b-9sszn_597c5e2d-af7f-4979-9175-d0f836161efc/maas-api/0.log" Jul 06 19:52:07.555240 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:07.555209 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m_c2780119-d899-424a-895e-9ef16b4c3cab/manager/0.log" Jul 06 19:52:07.614626 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:07.614588 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m_c2780119-d899-424a-895e-9ef16b4c3cab/manager/1.log" Jul 06 19:52:07.637374 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:07.637349 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58d5dcdfbc-6sb7m_c2780119-d899-424a-895e-9ef16b4c3cab/copy-manifests/0.log" Jul 06 19:52:07.711224 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:07.711194 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-nzrj5_0c7f699e-49b9-4132-8c15-b0046803b3ad/postgres/0.log" Jul 06 19:52:07.848162 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:07.848078 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5422w_1fa0868b-64da-4c83-b866-29bc1a9748ee/discovery/0.log" Jul 06 19:52:09.135912 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:09.135881 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-84979c6fd9-87xf5_61b53c91-e1dc-4f9f-942c-e9e6612cc4c7/manager/0.log" Jul 06 19:52:09.136335 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:09.135992 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-84979c6fd9-87xf5_61b53c91-e1dc-4f9f-942c-e9e6612cc4c7/manager/1.log" Jul 06 19:52:09.163324 ip-10-0-131-230 kubenswrapper[2557]: I0706 19:52:09.163280 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-xr8lb_75e93154-0e22-4ee5-8a96-a0ba96b3de29/openshift-lws-operator/0.log"