Jul 07 06:59:04.555902 ip-10-0-131-241 systemd[1]: Starting Kubernetes Kubelet... Jul 07 06:59:04.912761 ip-10-0-131-241 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 06:59:04.912761 ip-10-0-131-241 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 06:59:04.912761 ip-10-0-131-241 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 06:59:04.912761 ip-10-0-131-241 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 06:59:04.912761 ip-10-0-131-241 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 06:59:04.914626 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.914511 2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 06:59:04.921746 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921724 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 06:59:04.921746 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921744 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 06:59:04.921746 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921749 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 06:59:04.921746 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921754 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921758 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921761 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921764 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921767 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921770 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921772 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921775 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921777 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921780 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921783 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921785 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921788 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921791 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921794 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921796 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921799 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921801 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921811 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921814 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 06:59:04.921919 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921816 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921819 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921821 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921824 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921826 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921829 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921831 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921833 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921836 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921838 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921841 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921844 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921846 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921849 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921852 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921855 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921858 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921860 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921863 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921865 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 06:59:04.922444 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921868 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921871 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921873 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921875 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921878 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921881 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921883 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921887 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921890 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921892 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921895 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921898 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921901 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921903 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921906 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921908 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921911 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921913 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921915 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921918 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 06:59:04.922935 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921920 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921923 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921926 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921929 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921931 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921934 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921937 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921941 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921945 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921948 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921952 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921954 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921957 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921960 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921962 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921964 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921967 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921969 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921972 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 06:59:04.923422 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921974 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921985 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921988 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921991 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921993 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921996 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.921998 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922001 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922003 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922006 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922008 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922011 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922014 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922016 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922020 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922023 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922026 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922811 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922818 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 06:59:04.923887 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922822 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922825 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922828 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922831 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922834 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922836 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922839 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922842 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922846 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922850 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922853 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922855 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922858 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922860 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922863 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922867 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922870 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922873 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922875 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 06:59:04.924378 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922878 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922880 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922883 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922885 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922887 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922890 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922892 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922895 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922897 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922901 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922903 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922907 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922912 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922916 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922919 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922922 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922925 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922928 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922930 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922933 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 07 06:59:04.924873 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922935 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922938 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922941 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922944 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922946 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922949 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922952 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922954 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922957 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922960 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922963 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922965 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922968 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922970 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922973 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922975 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922978 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922981 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922984 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 06:59:04.925379 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922987 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922989 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922992 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922994 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.922997 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923000 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923002 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923005 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923007 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923010 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923012 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923015 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923017 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923019 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923023 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923026 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923028 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923031 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923033 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923037 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 06:59:04.925833 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923040 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923043 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923046 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923050 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923052 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923055 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923058 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923060 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923063 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923066 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923069 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923072 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923074 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923077 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923080 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923082 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923085 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923087 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923089 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923498 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 06:59:04.926334 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923503 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923507 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923510 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923513 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923515 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923518 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923520 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923524 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923528 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923531 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923533 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923536 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923539 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923541 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923544 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923549 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923551 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923554 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923557 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 06:59:04.926829 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923559 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923561 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923564 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923567 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923570 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923573 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923575 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923578 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923580 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923583 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923586 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923588 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923591 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923594 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923597 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923599 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923601 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923604 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923606 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 06:59:04.927292 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923609 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923611 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923615 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923618 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923621 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923624 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923627 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923629 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923631 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923634 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923640 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923643 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923645 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923648 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923650 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923653 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923656 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923660 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923664 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 06:59:04.927768 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923667 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923670 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923673 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923676 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923678 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923681 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923683 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923686 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923689 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923691 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923693 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923696 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923698 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923701 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923703 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923706 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923708 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923711 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923713 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923716 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 06:59:04.928222 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923718 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923720 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923723 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923725 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923730 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923733 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923735 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923738 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923741 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923744 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923747 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923749 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923752 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923754 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923757 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923759 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923762 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923764 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923767 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923769 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 06:59:04.928724 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.923771 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923848 2573 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923855 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923863 2573 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923871 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923877 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923883 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923889 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923893 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923897 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923900 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923903 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923909 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923912 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923915 2573 flags.go:64] FLAG: --cgroup-root="" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923918 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923921 2573 flags.go:64] FLAG: --client-ca-file="" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923925 2573 flags.go:64] FLAG: --cloud-provider="external" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923928 2573 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923935 2573 flags.go:64] FLAG: --cluster-domain="" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923942 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923945 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923948 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923951 2573 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 06:59:04.929204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923955 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923958 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923961 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923964 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923967 2573 flags.go:64] FLAG: --contention-profiling="false" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923970 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923973 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923976 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923979 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923983 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923986 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923989 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923992 2573 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923995 2573 flags.go:64] FLAG: --enable-server="true" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.923997 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924001 2573 flags.go:64] FLAG: --event-burst="100" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924004 2573 flags.go:64] FLAG: --event-qps="50" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924007 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924010 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924013 2573 flags.go:64] FLAG: --eviction-hard="" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924018 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924021 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924024 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924027 2573 flags.go:64] FLAG: --eviction-soft="" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924030 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 06:59:04.929785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924033 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924037 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924040 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924044 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924046 2573 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924049 2573 flags.go:64] FLAG: --feature-gates="" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924053 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924056 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924059 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924062 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924065 2573 flags.go:64] FLAG: --healthz-port="10248" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924068 2573 flags.go:64] FLAG: --help="false" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924071 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-131-241.ec2.internal" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924074 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924077 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924080 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924083 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924087 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924089 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924092 2573 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924096 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924099 2573 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924101 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924104 2573 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 06:59:04.930449 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924107 2573 flags.go:64] FLAG: --kube-reserved="" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924110 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924113 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924117 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924120 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924123 2573 flags.go:64] FLAG: --lock-file="" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924125 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924128 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924131 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924136 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924141 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924146 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924148 2573 flags.go:64] FLAG: --logging-format="text" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924151 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924154 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924157 2573 flags.go:64] FLAG: --manifest-url="" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924160 2573 flags.go:64] FLAG: --manifest-url-header="" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924164 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924167 2573 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924171 2573 flags.go:64] FLAG: --max-pods="110" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924174 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924177 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924179 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924183 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924186 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 06:59:04.931024 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924189 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924191 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924198 2573 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924201 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924204 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924207 2573 flags.go:64] FLAG: --pod-cidr="" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924210 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924215 2573 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924218 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924221 2573 flags.go:64] FLAG: --pods-per-core="0" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924226 2573 flags.go:64] FLAG: --port="10250" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924228 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924231 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09c01038ef82b6417" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924235 2573 flags.go:64] FLAG: --qos-reserved="" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924238 2573 flags.go:64] FLAG: --read-only-port="10255" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924240 2573 flags.go:64] FLAG: --register-node="true" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924243 2573 flags.go:64] FLAG: --register-with-taints="" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924247 2573 flags.go:64] FLAG: --registry-burst="10" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924250 2573 flags.go:64] FLAG: --registry-qps="5" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924254 2573 flags.go:64] FLAG: --reserved-cpus="" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924257 2573 flags.go:64] FLAG: --reserved-memory="" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924261 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924264 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924267 2573 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924270 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 06:59:04.931696 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924272 2573 flags.go:64] FLAG: --runonce="false" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924275 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924278 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924281 2573 flags.go:64] FLAG: --seccomp-default="false" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924283 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924286 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924289 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924292 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924295 2573 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924297 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924300 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924303 2573 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924306 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924309 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924311 2573 flags.go:64] FLAG: --system-cgroups="" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924314 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924319 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924335 2573 flags.go:64] FLAG: --tls-cert-file="" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924338 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924343 2573 flags.go:64] FLAG: --tls-min-version="" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924346 2573 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924349 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924352 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924355 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924358 2573 flags.go:64] FLAG: --v="2" Jul 07 06:59:04.932304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924361 2573 flags.go:64] FLAG: --version="false" Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924367 2573 flags.go:64] FLAG: --vmodule="" Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924371 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.924375 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924500 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924505 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924509 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924511 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924514 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924516 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924519 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924522 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924524 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924528 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924532 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924535 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924539 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924542 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924545 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 06:59:04.932918 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924548 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924550 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924553 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924555 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924558 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924562 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924565 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924567 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924570 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924572 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924575 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924578 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924580 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924583 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924587 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924590 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924592 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924594 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924597 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924600 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 06:59:04.933407 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924602 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924609 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924612 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924614 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924617 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924619 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924622 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924625 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924627 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924630 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924632 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924635 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924637 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924640 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924642 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924645 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924649 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924654 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924656 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924659 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 06:59:04.933933 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924662 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924664 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924666 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924669 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924672 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924674 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924679 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924681 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924684 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924687 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924690 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924692 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924695 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924697 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924700 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924703 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924705 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924708 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924710 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924713 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 06:59:04.934607 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924716 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924718 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924721 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924723 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924726 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924729 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924731 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924734 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924736 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924740 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924743 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924745 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924747 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924750 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924752 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924755 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924758 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924761 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924765 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924768 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 06:59:04.935095 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924770 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 06:59:04.935595 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924773 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 06:59:04.935595 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924775 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 06:59:04.935595 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.924778 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 06:59:04.935595 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.925548 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 06:59:04.935595 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.933763 2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 06:59:04.935595 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.933779 2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 06:59:04.935595 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.934724 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 06:59:04.935595 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.934732 2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 06:59:04.935595 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934794 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 06:59:04.935595 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934799 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 06:59:04.935595 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934802 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 06:59:04.935595 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934808 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 06:59:04.935595 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934811 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 06:59:04.935595 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934815 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934818 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934821 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934824 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934827 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934831 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934834 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934837 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934839 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934842 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934845 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934847 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934850 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934853 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934855 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934858 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934860 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934863 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934866 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934869 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 06:59:04.935947 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934872 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934874 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934877 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934879 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934882 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934885 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934888 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934890 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934893 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934896 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934898 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934901 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934904 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934906 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934909 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934912 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934914 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934917 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934919 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934922 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 06:59:04.936440 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934924 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934927 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934929 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934932 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934935 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934938 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934940 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934943 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934945 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934947 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934950 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934953 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934956 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934959 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934961 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934964 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934967 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934970 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934972 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 06:59:04.936909 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934976 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934978 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934980 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934983 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934986 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934988 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934991 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934993 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934996 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.934998 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935001 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935003 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935006 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935008 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935010 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935013 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935016 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935018 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935021 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935023 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 06:59:04.937390 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935026 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 06:59:04.937881 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935028 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 06:59:04.937881 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935031 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 06:59:04.937881 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935034 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 06:59:04.937881 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935036 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 06:59:04.937881 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935039 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 06:59:04.937881 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935042 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 06:59:04.937881 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935045 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 06:59:04.937881 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935049 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 06:59:04.937881 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935053 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 06:59:04.937881 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935056 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 06:59:04.937881 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935059 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 06:59:04.937881 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935061 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 06:59:04.937881 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935064 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 06:59:04.937881 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935067 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 06:59:04.938221 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.935071 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 06:59:04.938221 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935181 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 06:59:04.938221 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935185 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 06:59:04.938221 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935188 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 06:59:04.938221 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935192 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 06:59:04.938221 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935195 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 06:59:04.938221 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935197 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 06:59:04.938221 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935200 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 06:59:04.938221 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935202 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 06:59:04.938221 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935205 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 06:59:04.938221 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935207 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 06:59:04.938221 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935210 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 06:59:04.938221 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935214 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 06:59:04.938221 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935216 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 06:59:04.938221 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935219 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935221 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935224 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935226 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935229 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935232 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935234 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935236 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935239 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935242 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935245 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935247 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935250 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935253 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935255 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935258 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935260 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935263 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935265 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935268 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 06:59:04.938674 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935270 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935273 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935275 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935278 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935280 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935282 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935286 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935289 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935292 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935294 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935297 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935300 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935302 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935304 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935307 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935309 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935312 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935316 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935319 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 06:59:04.939156 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935336 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935338 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935341 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935344 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935347 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935350 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935353 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935356 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935359 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935362 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935365 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935367 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935370 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935372 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935375 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935377 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935380 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935382 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935385 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 06:59:04.939636 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935387 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935390 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935392 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935395 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935397 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935400 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935403 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935405 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935408 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935411 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935413 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935416 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935418 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935421 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935423 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935426 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935428 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935431 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935434 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 06:59:04.940083 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935437 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 06:59:04.940553 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935441 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 06:59:04.940553 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935443 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 06:59:04.940553 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935446 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 06:59:04.940553 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935448 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 06:59:04.940553 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935451 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 06:59:04.940553 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935453 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 06:59:04.940553 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935456 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 06:59:04.940553 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:04.935458 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 06:59:04.940553 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.935463 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 06:59:04.940553 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.935576 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 06:59:04.940553 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.937568 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 06:59:04.940553 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.938821 2573 server.go:1019] "Starting client certificate rotation" Jul 07 06:59:04.940553 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.938927 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 06:59:04.940553 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.938964 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 06:59:04.941587 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.941573 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 06:59:04.947521 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.947499 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 06:59:04.958937 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.958919 2573 log.go:25] "Validated CRI v1 runtime API" Jul 07 06:59:04.960793 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.960776 2573 log.go:25] "Validated CRI v1 image API" Jul 07 06:59:04.961649 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.961629 2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 06:59:04.965854 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.965810 2573 fs.go:135] Filesystem UUIDs: map[22a21c4b-bed5-4944-bb87-664f2c82bfbc:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c5260815-af76-41a3-b6bf-f20a948775c1:/dev/nvme0n1p3] Jul 07 06:59:04.965854 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.965834 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 06:59:04.969383 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.969361 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 06:59:04.970289 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.970175 2573 manager.go:216] Machine: {Timestamp:2026-07-07 06:59:04.969704015 +0000 UTC m=+0.305670323 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098179 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25df9992b41bfcbb7fafcd3f296dc3 SystemUUID:ec25df99-92b4-1bfc-bb7f-afcd3f296dc3 BootID:cf4d67a1-850a-4b9d-b522-7ec24ca44414 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:90:f8:f7:ee:cf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:90:f8:f7:ee:cf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:5b:66:9e:19:9c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 06:59:04.970289 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.970288 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 06:59:04.970414 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.970385 2573 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 06:59:04.974548 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.974521 2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 06:59:04.974688 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.974551 2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-241.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 06:59:04.975264 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.975254 2573 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 06:59:04.975294 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.975267 2573 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 06:59:04.975294 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.975275 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 06:59:04.975294 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.975286 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 06:59:04.976417 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.976406 2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 06:59:04.979054 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.979042 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 07 06:59:04.979162 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.979154 2573 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 06:59:04.979199 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.979185 2573 kubelet.go:486] "Attempting to sync node with API server" Jul 07 06:59:04.979199 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.979198 2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 06:59:04.979252 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.979208 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 06:59:04.979252 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.979217 2573 kubelet.go:398] "Adding apiserver pod source" Jul 07 06:59:04.979252 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.979225 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 06:59:04.979631 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.979616 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 06:59:04.979700 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.979641 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 06:59:04.981615 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.981599 2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 06:59:04.983129 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.983116 2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 06:59:04.983175 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.983147 2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 06:59:04.984243 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.984231 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 06:59:04.984283 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.984248 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 06:59:04.984283 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.984255 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 06:59:04.984283 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.984261 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 06:59:04.984283 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.984269 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 06:59:04.984283 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.984275 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 06:59:04.984283 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.984283 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 06:59:04.984458 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.984288 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 06:59:04.984458 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.984295 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 06:59:04.984458 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.984300 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 06:59:04.984458 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.984308 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 06:59:04.984458 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.984313 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 06:59:04.985412 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.985401 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 06:59:04.985412 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.985413 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 06:59:04.988492 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.988466 2573 server.go:1268] "Started kubelet" Jul 07 06:59:04.988574 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.988543 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 06:59:04.992029 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.991977 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 06:59:04.992968 ip-10-0-131-241 systemd[1]: Started Kubernetes Kubelet. Jul 07 06:59:04.993102 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:04.992996 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-241.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 06:59:04.993203 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.993183 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-241.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 06:59:04.993283 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:04.993230 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 06:59:04.993359 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.993343 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 06:59:04.995604 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.994509 2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 06:59:04.995604 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.994570 2573 server.go:310] "Adding debug handlers to kubelet server" Jul 07 06:59:04.997723 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:04.996839 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-241.ec2.internal.18bfefb698c99536 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-241.ec2.internal,UID:ip-10-0-131-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-241.ec2.internal,},FirstTimestamp:2026-07-07 06:59:04.988439862 +0000 UTC m=+0.324406170,LastTimestamp:2026-07-07 06:59:04.988439862 +0000 UTC m=+0.324406170,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-241.ec2.internal,}" Jul 07 06:59:04.998961 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:04.998945 2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 06:59:04.999969 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.999951 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 06:59:04.999969 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:04.999960 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 06:59:05.000104 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.000037 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:05.000499 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.000483 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 06:59:05.000499 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.000484 2573 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 06:59:05.000619 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.000510 2573 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 06:59:05.000619 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.000597 2573 reconstruct.go:97] "Volume reconstruction finished" Jul 07 06:59:05.000619 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.000605 2573 reconciler.go:29] "Reconciler: start to sync state" Jul 07 06:59:05.000830 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.000810 2573 factory.go:55] Registering systemd factory Jul 07 06:59:05.000926 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.000840 2573 factory.go:223] Registration of the systemd container factory successfully Jul 07 06:59:05.001047 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.001037 2573 factory.go:153] Registering CRI-O factory Jul 07 06:59:05.001085 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.001051 2573 factory.go:223] Registration of the crio container factory successfully Jul 07 06:59:05.001116 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.001100 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 06:59:05.001145 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.001117 2573 factory.go:103] Registering Raw factory Jul 07 06:59:05.001145 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.001130 2573 manager.go:1196] Started watching for new ooms in manager Jul 07 06:59:05.001560 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.001546 2573 manager.go:349] Starting recovery of all containers Jul 07 06:59:05.002241 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.002220 2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-241.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 06:59:05.002335 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.002302 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 06:59:05.003850 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.003825 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dgk2x" Jul 07 06:59:05.010032 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.010001 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dgk2x" Jul 07 06:59:05.011939 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.011921 2573 manager.go:354] Recovery completed Jul 07 06:59:05.016879 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.016863 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 06:59:05.018855 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.018838 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasSufficientMemory" Jul 07 06:59:05.018917 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.018872 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 06:59:05.018917 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.018886 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasSufficientPID" Jul 07 06:59:05.019742 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.019722 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 06:59:05.019742 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.019734 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 06:59:05.019869 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.019749 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 07 06:59:05.019869 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.019774 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 06:59:05.020442 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.020419 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasSufficientMemory" Jul 07 06:59:05.020515 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.020449 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 06:59:05.020515 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.020463 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasSufficientPID" Jul 07 06:59:05.020700 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.020679 2573 policy_none.go:49] "None policy: Start" Jul 07 06:59:05.020763 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.020706 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 06:59:05.020763 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.020720 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 06:59:05.021644 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.021631 2573 policy_none.go:47] "Start" Jul 07 06:59:05.052019 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.051865 2573 manager.go:337] "Starting Device Plugin manager" Jul 07 06:59:05.052118 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.052044 2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 06:59:05.052118 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.052059 2573 server.go:87] "Starting device plugin registration server" Jul 07 06:59:05.052310 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.052298 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 06:59:05.052389 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.052314 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 06:59:05.052441 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.052422 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 06:59:05.052529 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.052516 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 06:59:05.052578 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.052529 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 06:59:05.052982 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.052962 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 06:59:05.053041 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.053019 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:05.100220 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.100186 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 06:59:05.103919 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.101510 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 06:59:05.103919 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.101551 2573 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 06:59:05.103919 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.101573 2573 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 06:59:05.103919 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.101603 2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 06:59:05.104193 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.104175 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 06:59:05.152637 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.152617 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 06:59:05.153659 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.153643 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasSufficientMemory" Jul 07 06:59:05.153751 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.153675 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 06:59:05.153751 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.153693 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasSufficientPID" Jul 07 06:59:05.153751 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.153727 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.164412 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.164354 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.164412 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.164385 2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-241.ec2.internal\": node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:05.182545 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.182522 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:05.201710 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.201685 2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-241.ec2.internal"] Jul 07 06:59:05.201800 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.201785 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 06:59:05.203778 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.203760 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasSufficientMemory" Jul 07 06:59:05.203887 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.203791 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 06:59:05.203887 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.203802 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasSufficientPID" Jul 07 06:59:05.205010 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.204997 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 06:59:05.205148 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.205133 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.205187 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.205172 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 06:59:05.205753 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.205736 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasSufficientMemory" Jul 07 06:59:05.205824 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.205771 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 06:59:05.205824 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.205782 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasSufficientPID" Jul 07 06:59:05.205965 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.205736 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasSufficientMemory" Jul 07 06:59:05.205965 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.205848 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 06:59:05.205965 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.205861 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasSufficientPID" Jul 07 06:59:05.207272 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.207257 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.207332 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.207283 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 06:59:05.208035 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.208009 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasSufficientMemory" Jul 07 06:59:05.208121 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.208042 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 06:59:05.208121 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.208054 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeHasSufficientPID" Jul 07 06:59:05.231134 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.231113 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-241.ec2.internal\" not found" node="ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.235790 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.235770 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-241.ec2.internal\" not found" node="ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.283454 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.283428 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:05.302522 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.302495 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c175717ba8b5e711b25b7ec202a4efc9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal\" (UID: \"c175717ba8b5e711b25b7ec202a4efc9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.302598 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.302530 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c175717ba8b5e711b25b7ec202a4efc9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal\" (UID: \"c175717ba8b5e711b25b7ec202a4efc9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.302598 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.302551 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c079fdaed824bcba1a1cdfbf52df0d2c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-241.ec2.internal\" (UID: \"c079fdaed824bcba1a1cdfbf52df0d2c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.385796 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.385750 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:05.403416 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.403394 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c175717ba8b5e711b25b7ec202a4efc9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal\" (UID: \"c175717ba8b5e711b25b7ec202a4efc9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.403479 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.403422 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c175717ba8b5e711b25b7ec202a4efc9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal\" (UID: \"c175717ba8b5e711b25b7ec202a4efc9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.403479 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.403442 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c079fdaed824bcba1a1cdfbf52df0d2c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-241.ec2.internal\" (UID: \"c079fdaed824bcba1a1cdfbf52df0d2c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.403544 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.403491 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c079fdaed824bcba1a1cdfbf52df0d2c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-241.ec2.internal\" (UID: \"c079fdaed824bcba1a1cdfbf52df0d2c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.403544 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.403511 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c175717ba8b5e711b25b7ec202a4efc9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal\" (UID: \"c175717ba8b5e711b25b7ec202a4efc9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.403544 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.403517 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c175717ba8b5e711b25b7ec202a4efc9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal\" (UID: \"c175717ba8b5e711b25b7ec202a4efc9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.486080 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.485992 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:05.536167 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.536135 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.538915 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.538891 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-241.ec2.internal" Jul 07 06:59:05.587201 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.587161 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:05.693445 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.693416 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:05.794221 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.794129 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:05.899166 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.899130 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:05.939697 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.939656 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 06:59:05.940308 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:05.939865 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 06:59:05.999936 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:05.999886 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:06.012187 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.012131 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 06:54:05 +0000 UTC" deadline="2028-01-05 21:43:54.769373321 +0000 UTC" Jul 07 06:59:06.012187 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.012169 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13142h44m48.757208385s" Jul 07 06:59:06.100938 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:06.100834 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:06.122319 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:06.122285 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc175717ba8b5e711b25b7ec202a4efc9.slice/crio-4bd509f486de7542828311f376646fc7bb0d453ca8a6b028e5d440274bbc5cc9 WatchSource:0}: Error finding container 4bd509f486de7542828311f376646fc7bb0d453ca8a6b028e5d440274bbc5cc9: Status 404 returned error can't find the container with id 4bd509f486de7542828311f376646fc7bb0d453ca8a6b028e5d440274bbc5cc9 Jul 07 06:59:06.122564 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:06.122544 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc079fdaed824bcba1a1cdfbf52df0d2c.slice/crio-4681d337336f31d9d111a8cd2774f7d6b471df936417377a77c227db4cd2ece3 WatchSource:0}: Error finding container 4681d337336f31d9d111a8cd2774f7d6b471df936417377a77c227db4cd2ece3: Status 404 returned error can't find the container with id 4681d337336f31d9d111a8cd2774f7d6b471df936417377a77c227db4cd2ece3 Jul 07 06:59:06.127716 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.127697 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 06:59:06.164451 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.164420 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 06:59:06.201283 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:06.201240 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:06.219750 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.218280 2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 06:59:06.302365 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:06.302312 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:06.405267 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:06.405174 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:06.505403 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:06.505353 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-241.ec2.internal\" not found" Jul 07 06:59:06.542536 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.542506 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 06:59:06.600861 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.600827 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-241.ec2.internal" Jul 07 06:59:06.612604 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.612565 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 06:59:06.613685 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.613664 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal" Jul 07 06:59:06.625552 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.625526 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 06:59:06.981436 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.981399 2573 apiserver.go:52] "Watching apiserver" Jul 07 06:59:06.988573 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.988544 2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 06:59:06.991343 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.991290 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-xcg5n","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal","openshift-multus/multus-7wsjn","openshift-multus/multus-additional-cni-plugins-j9krq","openshift-multus/network-metrics-daemon-p79zw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg","openshift-dns/node-resolver-2g66m","openshift-image-registry/node-ca-bm5dq","openshift-network-diagnostics/network-check-target-x74xm","openshift-network-operator/iptables-alerter-sqd8s","openshift-ovn-kubernetes/ovnkube-node-gfdjz","kube-system/global-pull-secret-syncer-6zrvs","kube-system/konnectivity-agent-4stvb","kube-system/kube-apiserver-proxy-ip-10-0-131-241.ec2.internal"] Jul 07 06:59:06.994229 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.994207 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2g66m" Jul 07 06:59:06.997075 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.996388 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4stvb" Jul 07 06:59:06.997075 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.996933 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 06:59:06.997075 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.996938 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2gfzc\"" Jul 07 06:59:06.997075 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.996975 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 06:59:06.999039 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:06.999017 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.000705 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.000686 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 06:59:07.000808 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.000791 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 06:59:07.001316 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.001119 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.001316 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.001151 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 06:59:07.001898 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.001689 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-74p7f\"" Jul 07 06:59:07.001898 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.001705 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 06:59:07.001898 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.001725 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bv2tj\"" Jul 07 06:59:07.001898 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.001697 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 06:59:07.001898 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.001779 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 06:59:07.002969 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.002752 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 06:59:07.004578 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.004124 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:07.004578 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.004244 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 06:59:07.004578 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.004249 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g9xcn\"" Jul 07 06:59:07.004578 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.004238 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:07.004578 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.004460 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 06:59:07.006401 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.006376 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.009258 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.008959 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 06:59:07.009258 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.009108 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 06:59:07.009258 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.009180 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ttmv9\"" Jul 07 06:59:07.012738 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.010079 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:07.012738 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.010166 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:07.012738 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.010853 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 06:59:07.014006 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013204 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-cnibin\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.014006 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013251 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.014006 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013289 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98j8w\" (UniqueName: \"kubernetes.io/projected/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-kube-api-access-98j8w\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.014006 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013319 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-etc-kubernetes\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.014006 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013352 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-os-release\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.014006 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013383 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-multus-socket-dir-parent\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.014006 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013439 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a29da4ef-3d23-4022-bd07-dd84bae55b5e-hosts-file\") pod \"node-resolver-2g66m\" (UID: \"a29da4ef-3d23-4022-bd07-dd84bae55b5e\") " pod="openshift-dns/node-resolver-2g66m" Jul 07 06:59:07.014006 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013476 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2ec89b25-d95f-4f37-8486-9e6b81cdbaa7-agent-certs\") pod \"konnectivity-agent-4stvb\" (UID: \"2ec89b25-d95f-4f37-8486-9e6b81cdbaa7\") " pod="kube-system/konnectivity-agent-4stvb" Jul 07 06:59:07.014006 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013506 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-cnibin\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.014006 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013541 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-var-lib-cni-multus\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.014006 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013609 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-var-lib-kubelet\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.014006 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013647 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-multus-conf-dir\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.014006 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013727 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-run-multus-certs\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.014006 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013779 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.014006 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013823 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a29da4ef-3d23-4022-bd07-dd84bae55b5e-tmp-dir\") pod \"node-resolver-2g66m\" (UID: \"a29da4ef-3d23-4022-bd07-dd84bae55b5e\") " pod="openshift-dns/node-resolver-2g66m" Jul 07 06:59:07.014006 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013880 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-os-release\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.015044 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013928 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-multus-daemon-config\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.015044 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013959 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-system-cni-dir\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.015044 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.013993 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-cni-binary-copy\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.015044 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.014097 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-multus-cni-dir\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.015044 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.014161 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sxnv\" (UniqueName: \"kubernetes.io/projected/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-kube-api-access-2sxnv\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.015044 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.014274 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t98n\" (UniqueName: \"kubernetes.io/projected/a29da4ef-3d23-4022-bd07-dd84bae55b5e-kube-api-access-5t98n\") pod \"node-resolver-2g66m\" (UID: \"a29da4ef-3d23-4022-bd07-dd84bae55b5e\") " pod="openshift-dns/node-resolver-2g66m" Jul 07 06:59:07.015044 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.014367 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-var-lib-cni-bin\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.015044 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.014417 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-cni-binary-copy\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.015044 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.014457 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-run-k8s-cni-cncf-io\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.015044 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.014511 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-run-netns\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.015044 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.014553 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-hostroot\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.015044 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.014611 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.015044 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.014691 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2ec89b25-d95f-4f37-8486-9e6b81cdbaa7-konnectivity-ca\") pod \"konnectivity-agent-4stvb\" (UID: \"2ec89b25-d95f-4f37-8486-9e6b81cdbaa7\") " pod="kube-system/konnectivity-agent-4stvb" Jul 07 06:59:07.015044 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.014733 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-system-cni-dir\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.015930 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.015911 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.016028 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.015930 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 06:59:07.017964 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.017939 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bm5dq" Jul 07 06:59:07.018587 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.018566 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 06:59:07.018860 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.018845 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 06:59:07.018993 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.018845 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h9tgt\"" Jul 07 06:59:07.020620 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.020440 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 06:59:07.020729 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.020691 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-h92m2\"" Jul 07 06:59:07.020795 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.020767 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sqd8s" Jul 07 06:59:07.020908 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.020771 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 06:59:07.021171 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.021151 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 06:59:07.023303 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.023271 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.024388 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.024364 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 06:59:07.024709 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.024545 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 06:59:07.024797 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.024603 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 06:59:07.024797 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.024604 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tgtgq\"" Jul 07 06:59:07.025480 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.025461 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 06:59:07.026081 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.026059 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:07.026155 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.026137 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:07.026273 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.026239 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mkjsf\"" Jul 07 06:59:07.026650 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.026629 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 06:59:07.026764 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.026630 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 06:59:07.026764 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.026710 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 06:59:07.026764 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.026630 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 06:59:07.026973 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.026956 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 06:59:07.044507 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.044480 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2nnfm" Jul 07 06:59:07.053537 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.053504 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2nnfm" Jul 07 06:59:07.103163 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.103029 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 06:59:07.107148 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.107063 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal" event={"ID":"c175717ba8b5e711b25b7ec202a4efc9","Type":"ContainerStarted","Data":"4bd509f486de7542828311f376646fc7bb0d453ca8a6b028e5d440274bbc5cc9"} Jul 07 06:59:07.108026 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.107994 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-241.ec2.internal" event={"ID":"c079fdaed824bcba1a1cdfbf52df0d2c","Type":"ContainerStarted","Data":"4681d337336f31d9d111a8cd2774f7d6b471df936417377a77c227db4cd2ece3"} Jul 07 06:59:07.114891 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.114866 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-ovn-node-metrics-cert\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.114993 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.114906 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.114993 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.114927 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mdp2\" (UniqueName: \"kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2\") pod \"network-check-target-x74xm\" (UID: \"81061127-b732-4885-9f99-a9ad6c99301e\") " pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:07.114993 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.114943 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-host\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.114993 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.114981 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-multus-cni-dir\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.115185 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115031 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs\") pod \"network-metrics-daemon-p79zw\" (UID: \"672040eb-b7da-43ca-be5d-1a8f98a9dab7\") " pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:07.115185 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115058 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-slash\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.115185 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115075 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-env-overrides\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.115185 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115091 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-etc-selinux\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.115185 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115119 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sxnv\" (UniqueName: \"kubernetes.io/projected/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-kube-api-access-2sxnv\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.115185 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115166 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-multus-cni-dir\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.115473 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115185 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t98n\" (UniqueName: \"kubernetes.io/projected/a29da4ef-3d23-4022-bd07-dd84bae55b5e-kube-api-access-5t98n\") pod \"node-resolver-2g66m\" (UID: \"a29da4ef-3d23-4022-bd07-dd84bae55b5e\") " pod="openshift-dns/node-resolver-2g66m" Jul 07 06:59:07.115473 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115215 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-hostroot\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.115473 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115235 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-kubelet\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.115473 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115264 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-run-systemd\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.115473 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115291 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.115473 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115316 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-registration-dir\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.115473 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115350 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-hostroot\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.115473 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115354 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-sysctl-d\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.115473 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115406 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.115473 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115444 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-system-cni-dir\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.115473 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115471 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-cnibin\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.115909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115500 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98j8w\" (UniqueName: \"kubernetes.io/projected/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-kube-api-access-98j8w\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.115909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115533 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhbpz\" (UniqueName: \"kubernetes.io/projected/0c622a1f-4a29-45b7-a0de-785b67e27a5c-kube-api-access-dhbpz\") pod \"node-ca-bm5dq\" (UID: \"0c622a1f-4a29-45b7-a0de-785b67e27a5c\") " pod="openshift-image-registry/node-ca-bm5dq" Jul 07 06:59:07.115909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115555 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-system-cni-dir\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.115909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115560 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-var-lib-openvswitch\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.115909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115577 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-cnibin\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.115909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115607 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-cni-bin\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.115909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115631 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-cni-netd\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.115909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115668 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.115909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115673 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-multus-socket-dir-parent\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.115909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115753 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-multus-socket-dir-parent\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.115909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115796 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2ec89b25-d95f-4f37-8486-9e6b81cdbaa7-agent-certs\") pod \"konnectivity-agent-4stvb\" (UID: \"2ec89b25-d95f-4f37-8486-9e6b81cdbaa7\") " pod="kube-system/konnectivity-agent-4stvb" Jul 07 06:59:07.115909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115825 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-cnibin\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.115909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115846 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-var-lib-cni-multus\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.115909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115873 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0c622a1f-4a29-45b7-a0de-785b67e27a5c-serviceca\") pod \"node-ca-bm5dq\" (UID: \"0c622a1f-4a29-45b7-a0de-785b67e27a5c\") " pod="openshift-image-registry/node-ca-bm5dq" Jul 07 06:59:07.115909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115914 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t442z\" (UniqueName: \"kubernetes.io/projected/6b6f0398-67fc-4ef3-ae78-76de11e9ed00-kube-api-access-t442z\") pod \"iptables-alerter-sqd8s\" (UID: \"6b6f0398-67fc-4ef3-ae78-76de11e9ed00\") " pod="openshift-network-operator/iptables-alerter-sqd8s" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115923 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-var-lib-cni-multus\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115938 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-node-log\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.115966 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-log-socket\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116003 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-multus-conf-dir\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116012 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-cnibin\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116035 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-run-multus-certs\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116077 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-multus-conf-dir\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116120 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116150 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-run-multus-certs\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116201 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-run-netns\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116245 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-ovnkube-config\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116276 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-sysconfig\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116257 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116315 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-sysctl-conf\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116396 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-lib-modules\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116425 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a29da4ef-3d23-4022-bd07-dd84bae55b5e-tmp-dir\") pod \"node-resolver-2g66m\" (UID: \"a29da4ef-3d23-4022-bd07-dd84bae55b5e\") " pod="openshift-dns/node-resolver-2g66m" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116441 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-multus-daemon-config\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.116684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116529 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-cni-binary-copy\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116589 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0c622a1f-4a29-45b7-a0de-785b67e27a5c-host\") pod \"node-ca-bm5dq\" (UID: \"0c622a1f-4a29-45b7-a0de-785b67e27a5c\") " pod="openshift-image-registry/node-ca-bm5dq" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116719 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w68q2\" (UniqueName: \"kubernetes.io/projected/a8cea991-785c-4ea9-9169-820efef1a8c5-kube-api-access-w68q2\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116745 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a29da4ef-3d23-4022-bd07-dd84bae55b5e-tmp-dir\") pod \"node-resolver-2g66m\" (UID: \"a29da4ef-3d23-4022-bd07-dd84bae55b5e\") " pod="openshift-dns/node-resolver-2g66m" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116760 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116787 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx2dc\" (UniqueName: \"kubernetes.io/projected/8350fb8f-00ea-49e2-91c0-9af525c18ace-kube-api-access-fx2dc\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116814 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-modprobe-d\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116821 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116839 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-var-lib-kubelet\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116868 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3fde7f49-f438-445e-bfc1-d645d3157b33-kubelet-config\") pod \"global-pull-secret-syncer-6zrvs\" (UID: \"3fde7f49-f438-445e-bfc1-d645d3157b33\") " pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116908 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-var-lib-cni-bin\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116940 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b6f0398-67fc-4ef3-ae78-76de11e9ed00-host-slash\") pod \"iptables-alerter-sqd8s\" (UID: \"6b6f0398-67fc-4ef3-ae78-76de11e9ed00\") " pod="openshift-network-operator/iptables-alerter-sqd8s" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116969 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-etc-openvswitch\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.116994 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-systemd\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117024 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-cni-binary-copy\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117007 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-var-lib-cni-bin\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117070 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-run-k8s-cni-cncf-io\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.117547 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117097 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-multus-daemon-config\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117096 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-cni-binary-copy\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117100 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-run-netns\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117149 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-run-ovn\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117154 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-run-k8s-cni-cncf-io\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117177 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwwll\" (UniqueName: \"kubernetes.io/projected/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-kube-api-access-vwwll\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117225 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-sys-fs\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117253 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-tuned\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117181 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-run-netns\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117276 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8cea991-785c-4ea9-9169-820efef1a8c5-tmp\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117342 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2ec89b25-d95f-4f37-8486-9e6b81cdbaa7-konnectivity-ca\") pod \"konnectivity-agent-4stvb\" (UID: \"2ec89b25-d95f-4f37-8486-9e6b81cdbaa7\") " pod="kube-system/konnectivity-agent-4stvb" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117376 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117403 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-run-openvswitch\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117428 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-ovnkube-script-lib\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117452 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-socket-dir\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117489 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-sys\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117511 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-cni-binary-copy\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.118334 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117524 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-etc-kubernetes\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117563 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-etc-kubernetes\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117563 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-os-release\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117605 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a29da4ef-3d23-4022-bd07-dd84bae55b5e-hosts-file\") pod \"node-resolver-2g66m\" (UID: \"a29da4ef-3d23-4022-bd07-dd84bae55b5e\") " pod="openshift-dns/node-resolver-2g66m" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117623 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-os-release\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117631 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-run\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117656 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3fde7f49-f438-445e-bfc1-d645d3157b33-dbus\") pod \"global-pull-secret-syncer-6zrvs\" (UID: \"3fde7f49-f438-445e-bfc1-d645d3157b33\") " pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117675 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a29da4ef-3d23-4022-bd07-dd84bae55b5e-hosts-file\") pod \"node-resolver-2g66m\" (UID: \"a29da4ef-3d23-4022-bd07-dd84bae55b5e\") " pod="openshift-dns/node-resolver-2g66m" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117683 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-var-lib-kubelet\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117710 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6b6f0398-67fc-4ef3-ae78-76de11e9ed00-iptables-alerter-script\") pod \"iptables-alerter-sqd8s\" (UID: \"6b6f0398-67fc-4ef3-ae78-76de11e9ed00\") " pod="openshift-network-operator/iptables-alerter-sqd8s" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117735 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-systemd-units\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117751 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-host-var-lib-kubelet\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117761 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-device-dir\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117786 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-kubernetes\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117813 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret\") pod \"global-pull-secret-syncer-6zrvs\" (UID: \"3fde7f49-f438-445e-bfc1-d645d3157b33\") " pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117838 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-os-release\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117865 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-system-cni-dir\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.118901 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117881 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2ec89b25-d95f-4f37-8486-9e6b81cdbaa7-konnectivity-ca\") pod \"konnectivity-agent-4stvb\" (UID: \"2ec89b25-d95f-4f37-8486-9e6b81cdbaa7\") " pod="kube-system/konnectivity-agent-4stvb" Jul 07 06:59:07.119593 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117893 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fp6z\" (UniqueName: \"kubernetes.io/projected/672040eb-b7da-43ca-be5d-1a8f98a9dab7-kube-api-access-9fp6z\") pod \"network-metrics-daemon-p79zw\" (UID: \"672040eb-b7da-43ca-be5d-1a8f98a9dab7\") " pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:07.119593 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117898 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-os-release\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.119593 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117865 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.119593 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.117912 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-system-cni-dir\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.120991 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.120965 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2ec89b25-d95f-4f37-8486-9e6b81cdbaa7-agent-certs\") pod \"konnectivity-agent-4stvb\" (UID: \"2ec89b25-d95f-4f37-8486-9e6b81cdbaa7\") " pod="kube-system/konnectivity-agent-4stvb" Jul 07 06:59:07.124729 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.124711 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sxnv\" (UniqueName: \"kubernetes.io/projected/d28a6652-ec62-4431-8cbf-94ca2d9d0cfc-kube-api-access-2sxnv\") pod \"multus-7wsjn\" (UID: \"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc\") " pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.131469 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.131422 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t98n\" (UniqueName: \"kubernetes.io/projected/a29da4ef-3d23-4022-bd07-dd84bae55b5e-kube-api-access-5t98n\") pod \"node-resolver-2g66m\" (UID: \"a29da4ef-3d23-4022-bd07-dd84bae55b5e\") " pod="openshift-dns/node-resolver-2g66m" Jul 07 06:59:07.131909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.131875 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98j8w\" (UniqueName: \"kubernetes.io/projected/5bbe0aa3-493c-4233-b2a8-7feac9861c6f-kube-api-access-98j8w\") pod \"multus-additional-cni-plugins-j9krq\" (UID: \"5bbe0aa3-493c-4233-b2a8-7feac9861c6f\") " pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.218145 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218110 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-run-ovn\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.218319 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218160 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwwll\" (UniqueName: \"kubernetes.io/projected/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-kube-api-access-vwwll\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.218319 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218188 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-sys-fs\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.218319 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218194 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-run-ovn\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.218319 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218214 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-tuned\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.218319 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218234 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8cea991-785c-4ea9-9169-820efef1a8c5-tmp\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.218319 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218250 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-run-openvswitch\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.218319 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218269 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-ovnkube-script-lib\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.218319 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218286 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-socket-dir\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.218319 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218291 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-sys-fs\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.218319 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218303 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-sys\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.218319 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218322 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-run\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218340 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3fde7f49-f438-445e-bfc1-d645d3157b33-dbus\") pod \"global-pull-secret-syncer-6zrvs\" (UID: \"3fde7f49-f438-445e-bfc1-d645d3157b33\") " pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218375 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6b6f0398-67fc-4ef3-ae78-76de11e9ed00-iptables-alerter-script\") pod \"iptables-alerter-sqd8s\" (UID: \"6b6f0398-67fc-4ef3-ae78-76de11e9ed00\") " pod="openshift-network-operator/iptables-alerter-sqd8s" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218400 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-systemd-units\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218447 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-device-dir\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218469 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-kubernetes\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218493 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret\") pod \"global-pull-secret-syncer-6zrvs\" (UID: \"3fde7f49-f438-445e-bfc1-d645d3157b33\") " pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218522 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fp6z\" (UniqueName: \"kubernetes.io/projected/672040eb-b7da-43ca-be5d-1a8f98a9dab7-kube-api-access-9fp6z\") pod \"network-metrics-daemon-p79zw\" (UID: \"672040eb-b7da-43ca-be5d-1a8f98a9dab7\") " pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218549 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-ovn-node-metrics-cert\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218576 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218596 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdp2\" (UniqueName: \"kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2\") pod \"network-check-target-x74xm\" (UID: \"81061127-b732-4885-9f99-a9ad6c99301e\") " pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218611 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-host\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218627 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs\") pod \"network-metrics-daemon-p79zw\" (UID: \"672040eb-b7da-43ca-be5d-1a8f98a9dab7\") " pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218642 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-slash\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218694 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-env-overrides\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218710 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-etc-selinux\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218730 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-kubelet\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.218801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218746 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-run-systemd\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218768 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218820 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-registration-dir\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218850 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-sysctl-d\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218884 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhbpz\" (UniqueName: \"kubernetes.io/projected/0c622a1f-4a29-45b7-a0de-785b67e27a5c-kube-api-access-dhbpz\") pod \"node-ca-bm5dq\" (UID: \"0c622a1f-4a29-45b7-a0de-785b67e27a5c\") " pod="openshift-image-registry/node-ca-bm5dq" Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.218904 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218908 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-var-lib-openvswitch\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218952 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-cni-bin\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.218977 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs podName:672040eb-b7da-43ca-be5d-1a8f98a9dab7 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:07.71895459 +0000 UTC m=+3.054920909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs") pod "network-metrics-daemon-p79zw" (UID: "672040eb-b7da-43ca-be5d-1a8f98a9dab7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218975 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-run-openvswitch\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.218999 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-cni-bin\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219007 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-cni-netd\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219021 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-ovnkube-script-lib\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219044 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0c622a1f-4a29-45b7-a0de-785b67e27a5c-serviceca\") pod \"node-ca-bm5dq\" (UID: \"0c622a1f-4a29-45b7-a0de-785b67e27a5c\") " pod="openshift-image-registry/node-ca-bm5dq" Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219093 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219095 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-host\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219128 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-socket-dir\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.219508 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219135 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t442z\" (UniqueName: \"kubernetes.io/projected/6b6f0398-67fc-4ef3-ae78-76de11e9ed00-kube-api-access-t442z\") pod \"iptables-alerter-sqd8s\" (UID: \"6b6f0398-67fc-4ef3-ae78-76de11e9ed00\") " pod="openshift-network-operator/iptables-alerter-sqd8s" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219150 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-registration-dir\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219200 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219206 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-sys\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219255 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-node-log\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219266 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-run\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219268 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-kubelet\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219285 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-log-socket\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219316 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-run-netns\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219338 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3fde7f49-f438-445e-bfc1-d645d3157b33-dbus\") pod \"global-pull-secret-syncer-6zrvs\" (UID: \"3fde7f49-f438-445e-bfc1-d645d3157b33\") " pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219340 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-ovnkube-config\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219390 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-sysconfig\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219405 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-etc-selinux\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219421 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-sysctl-conf\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219441 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-env-overrides\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219457 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-sysctl-d\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219464 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-var-lib-openvswitch\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219496 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-slash\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.220038 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219595 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-sysctl-conf\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219702 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-lib-modules\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219735 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0c622a1f-4a29-45b7-a0de-785b67e27a5c-host\") pod \"node-ca-bm5dq\" (UID: \"0c622a1f-4a29-45b7-a0de-785b67e27a5c\") " pod="openshift-image-registry/node-ca-bm5dq" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219762 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w68q2\" (UniqueName: \"kubernetes.io/projected/a8cea991-785c-4ea9-9169-820efef1a8c5-kube-api-access-w68q2\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219793 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219814 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-systemd-units\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219821 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx2dc\" (UniqueName: \"kubernetes.io/projected/8350fb8f-00ea-49e2-91c0-9af525c18ace-kube-api-access-fx2dc\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219853 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-modprobe-d\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219881 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-var-lib-kubelet\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219902 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-ovnkube-config\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219908 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3fde7f49-f438-445e-bfc1-d645d3157b33-kubelet-config\") pod \"global-pull-secret-syncer-6zrvs\" (UID: \"3fde7f49-f438-445e-bfc1-d645d3157b33\") " pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219942 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0c622a1f-4a29-45b7-a0de-785b67e27a5c-serviceca\") pod \"node-ca-bm5dq\" (UID: \"0c622a1f-4a29-45b7-a0de-785b67e27a5c\") " pod="openshift-image-registry/node-ca-bm5dq" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219951 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3fde7f49-f438-445e-bfc1-d645d3157b33-kubelet-config\") pod \"global-pull-secret-syncer-6zrvs\" (UID: \"3fde7f49-f438-445e-bfc1-d645d3157b33\") " pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219966 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b6f0398-67fc-4ef3-ae78-76de11e9ed00-host-slash\") pod \"iptables-alerter-sqd8s\" (UID: \"6b6f0398-67fc-4ef3-ae78-76de11e9ed00\") " pod="openshift-network-operator/iptables-alerter-sqd8s" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219970 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6b6f0398-67fc-4ef3-ae78-76de11e9ed00-iptables-alerter-script\") pod \"iptables-alerter-sqd8s\" (UID: \"6b6f0398-67fc-4ef3-ae78-76de11e9ed00\") " pod="openshift-network-operator/iptables-alerter-sqd8s" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.219997 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-etc-openvswitch\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220004 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8350fb8f-00ea-49e2-91c0-9af525c18ace-device-dir\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.220816 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220005 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-run-systemd\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220027 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-sysconfig\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220040 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-cni-netd\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220072 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-var-lib-kubelet\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220088 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b6f0398-67fc-4ef3-ae78-76de11e9ed00-host-slash\") pod \"iptables-alerter-sqd8s\" (UID: \"6b6f0398-67fc-4ef3-ae78-76de11e9ed00\") " pod="openshift-network-operator/iptables-alerter-sqd8s" Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220079 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-etc-openvswitch\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220113 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-run-netns\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220113 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-lib-modules\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220139 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-node-log\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220152 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0c622a1f-4a29-45b7-a0de-785b67e27a5c-host\") pod \"node-ca-bm5dq\" (UID: \"0c622a1f-4a29-45b7-a0de-785b67e27a5c\") " pod="openshift-image-registry/node-ca-bm5dq" Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220142 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-kubernetes\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220163 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-log-socket\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220206 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.220258 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220315 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-systemd\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.220336 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret podName:3fde7f49-f438-445e-bfc1-d645d3157b33 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:07.720318907 +0000 UTC m=+3.056285207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret") pod "global-pull-secret-syncer-6zrvs" (UID: "3fde7f49-f438-445e-bfc1-d645d3157b33") : object "kube-system"/"original-pull-secret" not registered Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220426 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-systemd\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.221682 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.220544 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-modprobe-d\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.222607 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.221641 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8cea991-785c-4ea9-9169-820efef1a8c5-tmp\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.222607 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.222442 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8cea991-785c-4ea9-9169-820efef1a8c5-etc-tuned\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.222607 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.222564 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-ovn-node-metrics-cert\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.232970 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.232897 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fp6z\" (UniqueName: \"kubernetes.io/projected/672040eb-b7da-43ca-be5d-1a8f98a9dab7-kube-api-access-9fp6z\") pod \"network-metrics-daemon-p79zw\" (UID: \"672040eb-b7da-43ca-be5d-1a8f98a9dab7\") " pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:07.233479 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.233455 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwwll\" (UniqueName: \"kubernetes.io/projected/c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2-kube-api-access-vwwll\") pod \"ovnkube-node-gfdjz\" (UID: \"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.246067 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.246031 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 06:59:07.246067 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.246059 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 06:59:07.246067 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.246074 2573 projected.go:196] Error preparing data for projected volume kube-api-access-9mdp2 for pod openshift-network-diagnostics/network-check-target-x74xm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:59:07.246301 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.246145 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2 podName:81061127-b732-4885-9f99-a9ad6c99301e nodeName:}" failed. No retries permitted until 2026-07-07 06:59:07.746125621 +0000 UTC m=+3.082091943 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9mdp2" (UniqueName: "kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2") pod "network-check-target-x74xm" (UID: "81061127-b732-4885-9f99-a9ad6c99301e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:59:07.247436 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.247405 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w68q2\" (UniqueName: \"kubernetes.io/projected/a8cea991-785c-4ea9-9169-820efef1a8c5-kube-api-access-w68q2\") pod \"tuned-xcg5n\" (UID: \"a8cea991-785c-4ea9-9169-820efef1a8c5\") " pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.248193 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.247866 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhbpz\" (UniqueName: \"kubernetes.io/projected/0c622a1f-4a29-45b7-a0de-785b67e27a5c-kube-api-access-dhbpz\") pod \"node-ca-bm5dq\" (UID: \"0c622a1f-4a29-45b7-a0de-785b67e27a5c\") " pod="openshift-image-registry/node-ca-bm5dq" Jul 07 06:59:07.248312 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.248167 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx2dc\" (UniqueName: \"kubernetes.io/projected/8350fb8f-00ea-49e2-91c0-9af525c18ace-kube-api-access-fx2dc\") pod \"aws-ebs-csi-driver-node-m6fsg\" (UID: \"8350fb8f-00ea-49e2-91c0-9af525c18ace\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.256676 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.256650 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t442z\" (UniqueName: \"kubernetes.io/projected/6b6f0398-67fc-4ef3-ae78-76de11e9ed00-kube-api-access-t442z\") pod \"iptables-alerter-sqd8s\" (UID: \"6b6f0398-67fc-4ef3-ae78-76de11e9ed00\") " pod="openshift-network-operator/iptables-alerter-sqd8s" Jul 07 06:59:07.308452 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.308412 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2g66m" Jul 07 06:59:07.318424 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.318398 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4stvb" Jul 07 06:59:07.328342 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.328317 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7wsjn" Jul 07 06:59:07.335208 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.335187 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j9krq" Jul 07 06:59:07.343971 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.343946 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" Jul 07 06:59:07.351754 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.351732 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" Jul 07 06:59:07.361465 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.361446 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bm5dq" Jul 07 06:59:07.369281 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.369262 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sqd8s" Jul 07 06:59:07.375324 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.375308 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:07.498487 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.498412 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 06:59:07.723342 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.723297 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret\") pod \"global-pull-secret-syncer-6zrvs\" (UID: \"3fde7f49-f438-445e-bfc1-d645d3157b33\") " pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:07.723542 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.723379 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs\") pod \"network-metrics-daemon-p79zw\" (UID: \"672040eb-b7da-43ca-be5d-1a8f98a9dab7\") " pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:07.723542 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.723470 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 06:59:07.723542 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.723492 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:59:07.723542 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.723538 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret podName:3fde7f49-f438-445e-bfc1-d645d3157b33 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:08.72352098 +0000 UTC m=+4.059487277 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret") pod "global-pull-secret-syncer-6zrvs" (UID: "3fde7f49-f438-445e-bfc1-d645d3157b33") : object "kube-system"/"original-pull-secret" not registered Jul 07 06:59:07.723777 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.723559 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs podName:672040eb-b7da-43ca-be5d-1a8f98a9dab7 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:08.723548406 +0000 UTC m=+4.059514702 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs") pod "network-metrics-daemon-p79zw" (UID: "672040eb-b7da-43ca-be5d-1a8f98a9dab7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:59:07.824841 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:07.824753 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdp2\" (UniqueName: \"kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2\") pod \"network-check-target-x74xm\" (UID: \"81061127-b732-4885-9f99-a9ad6c99301e\") " pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:07.825030 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.824957 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 06:59:07.825030 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.824984 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 06:59:07.825030 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.824999 2573 projected.go:196] Error preparing data for projected volume kube-api-access-9mdp2 for pod openshift-network-diagnostics/network-check-target-x74xm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:59:07.825165 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:07.825076 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2 podName:81061127-b732-4885-9f99-a9ad6c99301e nodeName:}" failed. No retries permitted until 2026-07-07 06:59:08.825055484 +0000 UTC m=+4.161021797 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9mdp2" (UniqueName: "kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2") pod "network-check-target-x74xm" (UID: "81061127-b732-4885-9f99-a9ad6c99301e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:59:08.057270 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:08.057196 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 06:54:07 +0000 UTC" deadline="2028-01-11 00:13:07.994574989 +0000 UTC" Jul 07 06:59:08.057270 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:08.057266 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13265h13m59.937313678s" Jul 07 06:59:08.102626 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:08.102554 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:08.102763 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:08.102666 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:08.155562 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:08.155531 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc444a8a9_0d5b_45d0_953a_bc5f45c0b5b2.slice/crio-296c6328cca0721896dbe8dfdf39e16008be9f6ab6e3e6f570aae1a5bcbb764a WatchSource:0}: Error finding container 296c6328cca0721896dbe8dfdf39e16008be9f6ab6e3e6f570aae1a5bcbb764a: Status 404 returned error can't find the container with id 296c6328cca0721896dbe8dfdf39e16008be9f6ab6e3e6f570aae1a5bcbb764a Jul 07 06:59:08.156796 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:08.156771 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd28a6652_ec62_4431_8cbf_94ca2d9d0cfc.slice/crio-77da436cb1e1a68b667ac183744546ef795d4414cb64ee498ed1c9be38fcda23 WatchSource:0}: Error finding container 77da436cb1e1a68b667ac183744546ef795d4414cb64ee498ed1c9be38fcda23: Status 404 returned error can't find the container with id 77da436cb1e1a68b667ac183744546ef795d4414cb64ee498ed1c9be38fcda23 Jul 07 06:59:08.158352 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:08.157997 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8350fb8f_00ea_49e2_91c0_9af525c18ace.slice/crio-bb0cbad625851e542e8a3a309eefc3a2e1f27545fad0426195f948c868a5bbef WatchSource:0}: Error finding container bb0cbad625851e542e8a3a309eefc3a2e1f27545fad0426195f948c868a5bbef: Status 404 returned error can't find the container with id bb0cbad625851e542e8a3a309eefc3a2e1f27545fad0426195f948c868a5bbef Jul 07 06:59:08.158599 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:08.158369 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda29da4ef_3d23_4022_bd07_dd84bae55b5e.slice/crio-62372a2762bc5b297832240167fd1aa3c72c7d5c2cdd1eaeaa714acd0354502c WatchSource:0}: Error finding container 62372a2762bc5b297832240167fd1aa3c72c7d5c2cdd1eaeaa714acd0354502c: Status 404 returned error can't find the container with id 62372a2762bc5b297832240167fd1aa3c72c7d5c2cdd1eaeaa714acd0354502c Jul 07 06:59:08.161187 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:08.160672 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bbe0aa3_493c_4233_b2a8_7feac9861c6f.slice/crio-c6107125ce8de48f2a84b33610874ff9e7838a5af514ebd520542b729b007cd0 WatchSource:0}: Error finding container c6107125ce8de48f2a84b33610874ff9e7838a5af514ebd520542b729b007cd0: Status 404 returned error can't find the container with id c6107125ce8de48f2a84b33610874ff9e7838a5af514ebd520542b729b007cd0 Jul 07 06:59:08.162904 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:08.162797 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ec89b25_d95f_4f37_8486_9e6b81cdbaa7.slice/crio-64ede22a6db52fb7684d6e39324e523f1bdc4eb66d105c74dccb5d00ef1d2f54 WatchSource:0}: Error finding container 64ede22a6db52fb7684d6e39324e523f1bdc4eb66d105c74dccb5d00ef1d2f54: Status 404 returned error can't find the container with id 64ede22a6db52fb7684d6e39324e523f1bdc4eb66d105c74dccb5d00ef1d2f54 Jul 07 06:59:08.164539 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:08.163822 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b6f0398_67fc_4ef3_ae78_76de11e9ed00.slice/crio-328c3de8a002d65986b1def09076db91601c78f615501d912a54b9b36e153d4a WatchSource:0}: Error finding container 328c3de8a002d65986b1def09076db91601c78f615501d912a54b9b36e153d4a: Status 404 returned error can't find the container with id 328c3de8a002d65986b1def09076db91601c78f615501d912a54b9b36e153d4a Jul 07 06:59:08.165920 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:08.165699 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c622a1f_4a29_45b7_a0de_785b67e27a5c.slice/crio-fe81c377091cfa42349471859faaaf0aa6365973ca485311a00174218685728b WatchSource:0}: Error finding container fe81c377091cfa42349471859faaaf0aa6365973ca485311a00174218685728b: Status 404 returned error can't find the container with id fe81c377091cfa42349471859faaaf0aa6365973ca485311a00174218685728b Jul 07 06:59:08.735849 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:08.735502 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret\") pod \"global-pull-secret-syncer-6zrvs\" (UID: \"3fde7f49-f438-445e-bfc1-d645d3157b33\") " pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:08.735849 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:08.735824 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs\") pod \"network-metrics-daemon-p79zw\" (UID: \"672040eb-b7da-43ca-be5d-1a8f98a9dab7\") " pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:08.736083 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:08.735991 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:59:08.736083 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:08.736056 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs podName:672040eb-b7da-43ca-be5d-1a8f98a9dab7 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:10.736037054 +0000 UTC m=+6.072003366 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs") pod "network-metrics-daemon-p79zw" (UID: "672040eb-b7da-43ca-be5d-1a8f98a9dab7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:59:08.736512 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:08.736479 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 06:59:08.736621 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:08.736537 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret podName:3fde7f49-f438-445e-bfc1-d645d3157b33 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:10.7365206 +0000 UTC m=+6.072486902 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret") pod "global-pull-secret-syncer-6zrvs" (UID: "3fde7f49-f438-445e-bfc1-d645d3157b33") : object "kube-system"/"original-pull-secret" not registered Jul 07 06:59:08.836673 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:08.836630 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdp2\" (UniqueName: \"kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2\") pod \"network-check-target-x74xm\" (UID: \"81061127-b732-4885-9f99-a9ad6c99301e\") " pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:08.836873 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:08.836856 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 06:59:08.836949 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:08.836887 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 06:59:08.836949 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:08.836901 2573 projected.go:196] Error preparing data for projected volume kube-api-access-9mdp2 for pod openshift-network-diagnostics/network-check-target-x74xm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:59:08.837053 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:08.836973 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2 podName:81061127-b732-4885-9f99-a9ad6c99301e nodeName:}" failed. No retries permitted until 2026-07-07 06:59:10.836955011 +0000 UTC m=+6.172921311 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9mdp2" (UniqueName: "kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2") pod "network-check-target-x74xm" (UID: "81061127-b732-4885-9f99-a9ad6c99301e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:59:09.057719 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.057590 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 06:54:07 +0000 UTC" deadline="2027-12-04 20:40:32.875776947 +0000 UTC" Jul 07 06:59:09.057719 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.057630 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12373h41m23.818152834s" Jul 07 06:59:09.111524 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.108917 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:09.111524 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:09.109051 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:09.111524 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.109253 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:09.111524 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:09.109383 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:09.121617 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.121577 2573 generic.go:356] "Generic (PLEG): container finished" podID="c175717ba8b5e711b25b7ec202a4efc9" containerID="25d84408fcd850870ed8a03f7b491c5af760f482216562b1cdd92d630a18bcf6" exitCode=0 Jul 07 06:59:09.121780 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.121730 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal" event={"ID":"c175717ba8b5e711b25b7ec202a4efc9","Type":"ContainerDied","Data":"25d84408fcd850870ed8a03f7b491c5af760f482216562b1cdd92d630a18bcf6"} Jul 07 06:59:09.141418 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.141361 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-241.ec2.internal" event={"ID":"c079fdaed824bcba1a1cdfbf52df0d2c","Type":"ContainerStarted","Data":"bb616c8d475cb52ae1ec4e71be49783aabed8e237d0656ab51ace95933932b5c"} Jul 07 06:59:09.152691 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.152614 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" event={"ID":"a8cea991-785c-4ea9-9169-820efef1a8c5","Type":"ContainerStarted","Data":"7676165fe6093b61203e27c2996e09a5350007f819272679bd6951c0aa1089b8"} Jul 07 06:59:09.158430 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.158396 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bm5dq" event={"ID":"0c622a1f-4a29-45b7-a0de-785b67e27a5c","Type":"ContainerStarted","Data":"fe81c377091cfa42349471859faaaf0aa6365973ca485311a00174218685728b"} Jul 07 06:59:09.166824 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.166086 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-241.ec2.internal" podStartSLOduration=2.166056164 podStartE2EDuration="2.166056164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:59:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:09.1654478 +0000 UTC m=+4.501414126" watchObservedRunningTime="2026-07-07 06:59:09.166056164 +0000 UTC m=+4.502022488" Jul 07 06:59:09.181896 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.181746 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sqd8s" event={"ID":"6b6f0398-67fc-4ef3-ae78-76de11e9ed00","Type":"ContainerStarted","Data":"328c3de8a002d65986b1def09076db91601c78f615501d912a54b9b36e153d4a"} Jul 07 06:59:09.192515 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.192427 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2g66m" event={"ID":"a29da4ef-3d23-4022-bd07-dd84bae55b5e","Type":"ContainerStarted","Data":"62372a2762bc5b297832240167fd1aa3c72c7d5c2cdd1eaeaa714acd0354502c"} Jul 07 06:59:09.205464 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.205420 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4stvb" event={"ID":"2ec89b25-d95f-4f37-8486-9e6b81cdbaa7","Type":"ContainerStarted","Data":"64ede22a6db52fb7684d6e39324e523f1bdc4eb66d105c74dccb5d00ef1d2f54"} Jul 07 06:59:09.231389 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.231344 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j9krq" event={"ID":"5bbe0aa3-493c-4233-b2a8-7feac9861c6f","Type":"ContainerStarted","Data":"c6107125ce8de48f2a84b33610874ff9e7838a5af514ebd520542b729b007cd0"} Jul 07 06:59:09.249287 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.247829 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" event={"ID":"8350fb8f-00ea-49e2-91c0-9af525c18ace","Type":"ContainerStarted","Data":"bb0cbad625851e542e8a3a309eefc3a2e1f27545fad0426195f948c868a5bbef"} Jul 07 06:59:09.257350 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.257290 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7wsjn" event={"ID":"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc","Type":"ContainerStarted","Data":"77da436cb1e1a68b667ac183744546ef795d4414cb64ee498ed1c9be38fcda23"} Jul 07 06:59:09.276704 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:09.276650 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" event={"ID":"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2","Type":"ContainerStarted","Data":"296c6328cca0721896dbe8dfdf39e16008be9f6ab6e3e6f570aae1a5bcbb764a"} Jul 07 06:59:10.102179 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:10.102140 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:10.102721 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:10.102291 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:10.302919 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:10.302836 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal" event={"ID":"c175717ba8b5e711b25b7ec202a4efc9","Type":"ContainerStarted","Data":"d53fa021aa7c6e6e9648133929452c198a573300a296ac93c7de7508a31a1f85"} Jul 07 06:59:10.756066 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:10.756016 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret\") pod \"global-pull-secret-syncer-6zrvs\" (UID: \"3fde7f49-f438-445e-bfc1-d645d3157b33\") " pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:10.756280 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:10.756113 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs\") pod \"network-metrics-daemon-p79zw\" (UID: \"672040eb-b7da-43ca-be5d-1a8f98a9dab7\") " pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:10.756638 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:10.756310 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:59:10.756638 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:10.756395 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs podName:672040eb-b7da-43ca-be5d-1a8f98a9dab7 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:14.756358157 +0000 UTC m=+10.092324476 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs") pod "network-metrics-daemon-p79zw" (UID: "672040eb-b7da-43ca-be5d-1a8f98a9dab7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:59:10.756638 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:10.756474 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 06:59:10.756638 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:10.756516 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret podName:3fde7f49-f438-445e-bfc1-d645d3157b33 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:14.756503505 +0000 UTC m=+10.092469804 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret") pod "global-pull-secret-syncer-6zrvs" (UID: "3fde7f49-f438-445e-bfc1-d645d3157b33") : object "kube-system"/"original-pull-secret" not registered Jul 07 06:59:10.858260 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:10.857589 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdp2\" (UniqueName: \"kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2\") pod \"network-check-target-x74xm\" (UID: \"81061127-b732-4885-9f99-a9ad6c99301e\") " pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:10.858260 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:10.857804 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 06:59:10.858260 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:10.857826 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 06:59:10.858260 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:10.857840 2573 projected.go:196] Error preparing data for projected volume kube-api-access-9mdp2 for pod openshift-network-diagnostics/network-check-target-x74xm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:59:10.858260 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:10.857903 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2 podName:81061127-b732-4885-9f99-a9ad6c99301e nodeName:}" failed. No retries permitted until 2026-07-07 06:59:14.857884097 +0000 UTC m=+10.193850394 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9mdp2" (UniqueName: "kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2") pod "network-check-target-x74xm" (UID: "81061127-b732-4885-9f99-a9ad6c99301e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:59:11.105622 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:11.104814 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:11.105622 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:11.104956 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:11.105622 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:11.105378 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:11.105622 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:11.105501 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:12.102134 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:12.102099 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:12.102339 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:12.102263 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:13.102191 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:13.102153 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:13.102726 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:13.102292 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:13.102917 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:13.102842 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:13.102975 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:13.102942 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:14.102082 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:14.102045 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:14.102299 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:14.102215 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:14.794579 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:14.794538 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs\") pod \"network-metrics-daemon-p79zw\" (UID: \"672040eb-b7da-43ca-be5d-1a8f98a9dab7\") " pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:14.794782 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:14.794610 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret\") pod \"global-pull-secret-syncer-6zrvs\" (UID: \"3fde7f49-f438-445e-bfc1-d645d3157b33\") " pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:14.794782 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:14.794722 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 06:59:14.794782 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:14.794736 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:59:14.794953 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:14.794782 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret podName:3fde7f49-f438-445e-bfc1-d645d3157b33 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:22.794764188 +0000 UTC m=+18.130730486 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret") pod "global-pull-secret-syncer-6zrvs" (UID: "3fde7f49-f438-445e-bfc1-d645d3157b33") : object "kube-system"/"original-pull-secret" not registered Jul 07 06:59:14.794953 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:14.794814 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs podName:672040eb-b7da-43ca-be5d-1a8f98a9dab7 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:22.794792464 +0000 UTC m=+18.130758786 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs") pod "network-metrics-daemon-p79zw" (UID: "672040eb-b7da-43ca-be5d-1a8f98a9dab7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:59:14.896128 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:14.895806 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdp2\" (UniqueName: \"kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2\") pod \"network-check-target-x74xm\" (UID: \"81061127-b732-4885-9f99-a9ad6c99301e\") " pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:14.896128 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:14.896001 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 06:59:14.896128 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:14.896022 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 06:59:14.896128 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:14.896036 2573 projected.go:196] Error preparing data for projected volume kube-api-access-9mdp2 for pod openshift-network-diagnostics/network-check-target-x74xm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:59:14.896128 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:14.896104 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2 podName:81061127-b732-4885-9f99-a9ad6c99301e nodeName:}" failed. No retries permitted until 2026-07-07 06:59:22.896085098 +0000 UTC m=+18.232051398 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9mdp2" (UniqueName: "kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2") pod "network-check-target-x74xm" (UID: "81061127-b732-4885-9f99-a9ad6c99301e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:59:15.103639 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:15.103082 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:15.103639 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:15.103206 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:15.103639 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:15.103218 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:15.103639 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:15.103335 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:16.102193 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:16.102135 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:16.102410 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:16.102288 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:17.102535 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:17.102344 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:17.102535 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:17.102403 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:17.102535 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:17.102499 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:17.103165 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:17.102655 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:18.102623 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:18.102097 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:18.102623 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:18.102253 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:19.102073 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:19.102035 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:19.102241 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:19.102035 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:19.102241 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:19.102168 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:19.102241 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:19.102207 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:20.101828 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:20.101795 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:20.102289 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:20.101939 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:21.101846 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:21.101797 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:21.102304 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:21.101930 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:21.102304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:21.101967 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:21.102304 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:21.102039 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:22.102413 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:22.102380 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:22.102901 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:22.102513 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:22.860751 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:22.860717 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret\") pod \"global-pull-secret-syncer-6zrvs\" (UID: \"3fde7f49-f438-445e-bfc1-d645d3157b33\") " pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:22.861014 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:22.860769 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs\") pod \"network-metrics-daemon-p79zw\" (UID: \"672040eb-b7da-43ca-be5d-1a8f98a9dab7\") " pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:22.861014 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:22.860881 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 06:59:22.861014 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:22.860940 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret podName:3fde7f49-f438-445e-bfc1-d645d3157b33 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:38.8609258 +0000 UTC m=+34.196892094 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret") pod "global-pull-secret-syncer-6zrvs" (UID: "3fde7f49-f438-445e-bfc1-d645d3157b33") : object "kube-system"/"original-pull-secret" not registered Jul 07 06:59:22.861014 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:22.860889 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:59:22.861216 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:22.861022 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs podName:672040eb-b7da-43ca-be5d-1a8f98a9dab7 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:38.861005371 +0000 UTC m=+34.196971667 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs") pod "network-metrics-daemon-p79zw" (UID: "672040eb-b7da-43ca-be5d-1a8f98a9dab7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:59:22.961043 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:22.961010 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdp2\" (UniqueName: \"kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2\") pod \"network-check-target-x74xm\" (UID: \"81061127-b732-4885-9f99-a9ad6c99301e\") " pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:22.961250 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:22.961190 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 06:59:22.961250 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:22.961212 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 06:59:22.961250 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:22.961223 2573 projected.go:196] Error preparing data for projected volume kube-api-access-9mdp2 for pod openshift-network-diagnostics/network-check-target-x74xm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:59:22.961408 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:22.961274 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2 podName:81061127-b732-4885-9f99-a9ad6c99301e nodeName:}" failed. No retries permitted until 2026-07-07 06:59:38.961258681 +0000 UTC m=+34.297224976 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9mdp2" (UniqueName: "kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2") pod "network-check-target-x74xm" (UID: "81061127-b732-4885-9f99-a9ad6c99301e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:59:23.102498 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:23.102461 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:23.102999 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:23.102470 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:23.102999 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:23.102599 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:23.102999 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:23.102669 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:24.101931 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:24.101898 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:24.102111 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:24.102032 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:25.103337 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:25.103285 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:25.103751 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:25.103423 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:25.103751 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:25.103482 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:25.103751 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:25.103593 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:26.102442 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:26.102409 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:26.102603 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:26.102532 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:27.102040 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:27.102004 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:27.102472 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:27.102004 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:27.102472 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:27.102124 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:27.102472 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:27.102192 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:28.102915 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.102498 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:28.103495 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:28.103039 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:28.338865 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.338782 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4stvb" event={"ID":"2ec89b25-d95f-4f37-8486-9e6b81cdbaa7","Type":"ContainerStarted","Data":"f921d69cca6bc2123d30f3ee4c8fd1e1c4c2ad6393f48ea749d1a8f156ce0006"} Jul 07 06:59:28.340316 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.340291 2573 generic.go:356] "Generic (PLEG): container finished" podID="5bbe0aa3-493c-4233-b2a8-7feac9861c6f" containerID="d2a067b12f92786cf1f721bca9193249b1d6ed033be3935b6b7a706748bf4376" exitCode=0 Jul 07 06:59:28.340452 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.340348 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j9krq" event={"ID":"5bbe0aa3-493c-4233-b2a8-7feac9861c6f","Type":"ContainerDied","Data":"d2a067b12f92786cf1f721bca9193249b1d6ed033be3935b6b7a706748bf4376"} Jul 07 06:59:28.341950 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.341927 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" event={"ID":"8350fb8f-00ea-49e2-91c0-9af525c18ace","Type":"ContainerStarted","Data":"28bdaa9c01037aa2740d3735d10bd51f76dfa690f63c7ab0f591b1204b2b63d4"} Jul 07 06:59:28.344018 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.343992 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7wsjn" event={"ID":"d28a6652-ec62-4431-8cbf-94ca2d9d0cfc","Type":"ContainerStarted","Data":"8fa1cb4b5aec2132c9e10c3be7c5f22f1a9465610da48b7983d45904a7caeb8f"} Jul 07 06:59:28.348903 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.348881 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" event={"ID":"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2","Type":"ContainerStarted","Data":"89f45253b9dc29af4d9cea7ef1351d1ebe180de163b330e2cd8973696d17f359"} Jul 07 06:59:28.348976 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.348941 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" event={"ID":"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2","Type":"ContainerStarted","Data":"94129458f26d17d297d3c6e5117f3e8c53a943f5af8d42e4f06aeb2102845930"} Jul 07 06:59:28.348976 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.348972 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" event={"ID":"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2","Type":"ContainerStarted","Data":"f4fc2a7e3c3b9afcb4d1afbc37407c2fecfe9fe0177db8205eff6f8b3fa3483c"} Jul 07 06:59:28.349059 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.349009 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" event={"ID":"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2","Type":"ContainerStarted","Data":"db691a94c72ccfc07a340e4519190cf9480b9ffa2c3b9540f55f61fdf78a0af0"} Jul 07 06:59:28.349059 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.349037 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" event={"ID":"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2","Type":"ContainerStarted","Data":"ca4e12154096868ceaab5eeb20e075211f7b4185997970f4b18fe23899d1c857"} Jul 07 06:59:28.349140 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.349069 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" event={"ID":"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2","Type":"ContainerStarted","Data":"4d9e93463877b68680875e1ce16c28c020858612e1b81ae9efdd9a2e8780f5ec"} Jul 07 06:59:28.350234 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.350214 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" event={"ID":"a8cea991-785c-4ea9-9169-820efef1a8c5","Type":"ContainerStarted","Data":"228d09089c012a1c426d92fe4cc7860dde58350d3c93431c0577b0f9ef1b0a3b"} Jul 07 06:59:28.351431 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.351412 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bm5dq" event={"ID":"0c622a1f-4a29-45b7-a0de-785b67e27a5c","Type":"ContainerStarted","Data":"2ad8ab9630116afaffd9e10a333f2c976c63b87d498158f673a3d115b8e7a2db"} Jul 07 06:59:28.352629 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.352610 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2g66m" event={"ID":"a29da4ef-3d23-4022-bd07-dd84bae55b5e","Type":"ContainerStarted","Data":"b8db6fa3bb1b5618127eab7e4884df8861f13603d4d3bf8706ae7f3bcbcea9dc"} Jul 07 06:59:28.359695 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.359663 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4stvb" podStartSLOduration=10.99103799 podStartE2EDuration="22.359651827s" totalImagesPullingTime="11.368613837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:28.359243346 +0000 UTC m=+23.695209683" watchObservedRunningTime="2026-07-07 06:59:28.359651827 +0000 UTC m=+23.695618144" Jul 07 06:59:28.359809 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.359791 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-241.ec2.internal" podStartSLOduration=21.35978549 podStartE2EDuration="21.35978549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:59:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:10.331139309 +0000 UTC m=+5.667105630" watchObservedRunningTime="2026-07-07 06:59:28.35978549 +0000 UTC m=+23.695751806" Jul 07 06:59:28.376478 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.376433 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2g66m" podStartSLOduration=2.988158699 podStartE2EDuration="22.376411316s" totalImagesPullingTime="19.388252617s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:28.376403589 +0000 UTC m=+23.712369908" watchObservedRunningTime="2026-07-07 06:59:28.376411316 +0000 UTC m=+23.712377627" Jul 07 06:59:28.419487 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.419437 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xcg5n" podStartSLOduration=3.039464948 podStartE2EDuration="22.419423936s" totalImagesPullingTime="19.379958988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:28.419342944 +0000 UTC m=+23.755309286" watchObservedRunningTime="2026-07-07 06:59:28.419423936 +0000 UTC m=+23.755390252" Jul 07 06:59:28.439237 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.439192 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bm5dq" podStartSLOduration=3.05746672 podStartE2EDuration="22.439178545s" totalImagesPullingTime="19.381711825s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:28.439055087 +0000 UTC m=+23.775021404" watchObservedRunningTime="2026-07-07 06:59:28.439178545 +0000 UTC m=+23.775144862" Jul 07 06:59:28.462833 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.462781 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7wsjn" podStartSLOduration=3.008736743 podStartE2EDuration="22.462763112s" totalImagesPullingTime="19.454026369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:28.462619114 +0000 UTC m=+23.798585443" watchObservedRunningTime="2026-07-07 06:59:28.462763112 +0000 UTC m=+23.798729432" Jul 07 06:59:28.564778 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.564748 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:28.657132 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.657101 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:28.758793 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:28.758740 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:29.102243 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:29.102210 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:29.102385 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:29.102210 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:29.102385 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:29.102357 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:29.102514 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:29.102422 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:29.240359 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:29.240334 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 06:59:29.356558 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:29.356489 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sqd8s" event={"ID":"6b6f0398-67fc-4ef3-ae78-76de11e9ed00","Type":"ContainerStarted","Data":"c24670b8c412c5f614293ab4883e0fde64012125bacd4e1500cba37c0993ef6f"} Jul 07 06:59:29.358702 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:29.358658 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" event={"ID":"8350fb8f-00ea-49e2-91c0-9af525c18ace","Type":"ContainerStarted","Data":"e54ed4842fc591c0398828f1539fe9fdd3fc00bb50339a253c2d9991652277a9"} Jul 07 06:59:29.375293 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:29.375251 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sqd8s" podStartSLOduration=3.992828705 podStartE2EDuration="23.375239183s" totalImagesPullingTime="19.382410478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:29.37503732 +0000 UTC m=+24.711003636" watchObservedRunningTime="2026-07-07 06:59:29.375239183 +0000 UTC m=+24.711205499" Jul 07 06:59:29.508110 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:29.508082 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:29.593065 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:29.593035 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:30.064113 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:30.064019 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T06:59:29.240354755Z","UUID":"2c215793-ec6a-44dd-8d5c-7d6d129e1c94","Handler":null,"Name":"","Endpoint":""} Jul 07 06:59:30.066011 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:30.065989 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 06:59:30.066151 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:30.066044 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 06:59:30.102310 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:30.102279 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:30.102455 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:30.102437 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:30.365807 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:30.365770 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" event={"ID":"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2","Type":"ContainerStarted","Data":"a1b49a2646d513d1d48884109bdf90a02bd7949ba5dc2b2d834342283913af93"} Jul 07 06:59:30.489317 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:30.489286 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4stvb" Jul 07 06:59:30.489822 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:30.489803 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4stvb" Jul 07 06:59:31.102055 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:31.102029 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:31.102308 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:31.102028 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:31.102308 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:31.102138 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:31.102308 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:31.102234 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:31.371060 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:31.370811 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" event={"ID":"8350fb8f-00ea-49e2-91c0-9af525c18ace","Type":"ContainerStarted","Data":"50c22eba628c94907ff7542ec0cc572fca4062064087338c6ed3761b88c91629"} Jul 07 06:59:31.371531 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:31.371143 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4stvb" Jul 07 06:59:31.371717 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:31.371701 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4stvb" Jul 07 06:59:31.393900 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:31.393857 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6fsg" podStartSLOduration=3.466766431 podStartE2EDuration="25.393839445s" totalImagesPullingTime="21.927073014s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:59:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:31.393613443 +0000 UTC m=+26.729579759" watchObservedRunningTime="2026-07-07 06:59:31.393839445 +0000 UTC m=+26.729805761" Jul 07 06:59:31.468933 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:31.468894 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:32.102611 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:32.102588 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:32.102763 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:32.102688 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:33.101767 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:33.101738 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:33.102705 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:33.101870 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:33.102705 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:33.101867 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:33.102705 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:33.101949 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:33.379807 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:33.379575 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" event={"ID":"c444a8a9-0d5b-45d0-953a-bc5f45c0b5b2","Type":"ContainerStarted","Data":"6e0ad60317897f67d127c21fbc3a3eaf5b4892ee9bd39466745052ae71ec3d4c"} Jul 07 06:59:33.414354 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:33.414230 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" podStartSLOduration=7.951823163 podStartE2EDuration="27.414210255s" totalImagesPullingTime="19.462387092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:33.412984131 +0000 UTC m=+28.748950487" watchObservedRunningTime="2026-07-07 06:59:33.414210255 +0000 UTC m=+28.750176574" Jul 07 06:59:34.102157 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:34.102122 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:34.102561 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:34.102272 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:34.382755 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:34.382530 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:34.382755 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:34.382614 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:34.382755 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:34.382657 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:34.397181 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:34.397157 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:34.398588 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:34.398568 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 06:59:34.850283 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:34.850252 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x74xm"] Jul 07 06:59:34.850430 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:34.850390 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:34.850508 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:34.850487 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:34.851494 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:34.851473 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6zrvs"] Jul 07 06:59:34.851607 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:34.851551 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:34.851673 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:34.851618 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:34.866121 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:34.866091 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p79zw"] Jul 07 06:59:34.866225 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:34.866168 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:34.866288 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:34.866266 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:35.386148 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:35.386106 2573 generic.go:356] "Generic (PLEG): container finished" podID="5bbe0aa3-493c-4233-b2a8-7feac9861c6f" containerID="f19f3e6ab50f216db25a834a5eefb18f0c1b140711db6ae70311fc1996819861" exitCode=0 Jul 07 06:59:35.386645 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:35.386201 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j9krq" event={"ID":"5bbe0aa3-493c-4233-b2a8-7feac9861c6f","Type":"ContainerDied","Data":"f19f3e6ab50f216db25a834a5eefb18f0c1b140711db6ae70311fc1996819861"} Jul 07 06:59:36.102012 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:36.101981 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:36.102185 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:36.102032 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:36.102185 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:36.102148 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:36.102270 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:36.102247 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:36.390749 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:36.390680 2573 generic.go:356] "Generic (PLEG): container finished" podID="5bbe0aa3-493c-4233-b2a8-7feac9861c6f" containerID="ad98d6b2ce76dc4ea8b88398498a355c110ef7a70d316394eb2f6879479ff6ec" exitCode=0 Jul 07 06:59:36.390749 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:36.390716 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j9krq" event={"ID":"5bbe0aa3-493c-4233-b2a8-7feac9861c6f","Type":"ContainerDied","Data":"ad98d6b2ce76dc4ea8b88398498a355c110ef7a70d316394eb2f6879479ff6ec"} Jul 07 06:59:37.102100 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:37.102068 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:37.102297 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:37.102174 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:37.395592 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:37.395518 2573 generic.go:356] "Generic (PLEG): container finished" podID="5bbe0aa3-493c-4233-b2a8-7feac9861c6f" containerID="c101c5a2a49ae4baa399498515c8abc9ec7b84ce299e8c5f40e71c9453734c7d" exitCode=0 Jul 07 06:59:37.395592 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:37.395556 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j9krq" event={"ID":"5bbe0aa3-493c-4233-b2a8-7feac9861c6f","Type":"ContainerDied","Data":"c101c5a2a49ae4baa399498515c8abc9ec7b84ce299e8c5f40e71c9453734c7d"} Jul 07 06:59:38.101978 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:38.101933 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:38.101978 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:38.101968 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:38.102201 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:38.102074 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:38.102261 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:38.102216 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:38.882828 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:38.882790 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret\") pod \"global-pull-secret-syncer-6zrvs\" (UID: \"3fde7f49-f438-445e-bfc1-d645d3157b33\") " pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:38.883411 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:38.882844 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs\") pod \"network-metrics-daemon-p79zw\" (UID: \"672040eb-b7da-43ca-be5d-1a8f98a9dab7\") " pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:38.883411 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:38.882936 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 06:59:38.883411 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:38.882959 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:59:38.883411 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:38.882998 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret podName:3fde7f49-f438-445e-bfc1-d645d3157b33 nodeName:}" failed. No retries permitted until 2026-07-07 07:00:10.882983247 +0000 UTC m=+66.218949546 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret") pod "global-pull-secret-syncer-6zrvs" (UID: "3fde7f49-f438-445e-bfc1-d645d3157b33") : object "kube-system"/"original-pull-secret" not registered Jul 07 06:59:38.883411 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:38.883012 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs podName:672040eb-b7da-43ca-be5d-1a8f98a9dab7 nodeName:}" failed. No retries permitted until 2026-07-07 07:00:10.883005852 +0000 UTC m=+66.218972147 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs") pod "network-metrics-daemon-p79zw" (UID: "672040eb-b7da-43ca-be5d-1a8f98a9dab7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:59:38.983695 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:38.983662 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdp2\" (UniqueName: \"kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2\") pod \"network-check-target-x74xm\" (UID: \"81061127-b732-4885-9f99-a9ad6c99301e\") " pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:38.983876 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:38.983855 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 06:59:38.983934 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:38.983882 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 06:59:38.983934 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:38.983895 2573 projected.go:196] Error preparing data for projected volume kube-api-access-9mdp2 for pod openshift-network-diagnostics/network-check-target-x74xm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:59:38.984025 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:38.983949 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2 podName:81061127-b732-4885-9f99-a9ad6c99301e nodeName:}" failed. No retries permitted until 2026-07-07 07:00:10.983936043 +0000 UTC m=+66.319902338 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9mdp2" (UniqueName: "kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2") pod "network-check-target-x74xm" (UID: "81061127-b732-4885-9f99-a9ad6c99301e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:59:39.102382 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:39.102358 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:39.102561 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:39.102454 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zrvs" podUID="3fde7f49-f438-445e-bfc1-d645d3157b33" Jul 07 06:59:40.102580 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.102553 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:40.102580 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.102556 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:40.103017 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:40.102700 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x74xm" podUID="81061127-b732-4885-9f99-a9ad6c99301e" Jul 07 06:59:40.103017 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:40.102836 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p79zw" podUID="672040eb-b7da-43ca-be5d-1a8f98a9dab7" Jul 07 06:59:40.511281 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.511248 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-241.ec2.internal" event="NodeReady" Jul 07 06:59:40.511458 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.511410 2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 06:59:40.567188 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.567155 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pwjq8"] Jul 07 06:59:40.571757 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.571725 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dw7c2"] Jul 07 06:59:40.571895 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.571860 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:40.574542 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.574507 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 06:59:40.574669 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.574573 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vfjqn\"" Jul 07 06:59:40.574786 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.574766 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 06:59:40.575343 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.575311 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dw7c2" Jul 07 06:59:40.579680 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.579623 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7mlnr\"" Jul 07 06:59:40.579885 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.579868 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 06:59:40.579953 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.579913 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 06:59:40.580010 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.579877 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 06:59:40.583849 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.583810 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pwjq8"] Jul 07 06:59:40.589465 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.589440 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dw7c2"] Jul 07 06:59:40.592986 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.592968 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-j2872"] Jul 07 06:59:40.596855 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.596558 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.597551 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.597531 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-ndp4z"] Jul 07 06:59:40.600052 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.600033 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 06:59:40.600281 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.600264 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 06:59:40.600589 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.600466 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 06:59:40.600589 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.600466 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 06:59:40.600589 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.600555 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ckrdm\"" Jul 07 06:59:40.600863 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.600795 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx"] Jul 07 06:59:40.600935 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.600886 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 06:59:40.604181 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.604163 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 06:59:40.604626 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.604608 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 06:59:40.605290 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.605274 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 06:59:40.605430 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.605354 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 06:59:40.605527 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.605482 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 06:59:40.606462 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.606442 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 06:59:40.606777 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.606761 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-4s5nm\"" Jul 07 06:59:40.607918 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.607894 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 06:59:40.608274 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.608254 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 06:59:40.608451 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.608304 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 06:59:40.608451 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.608402 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 06:59:40.608563 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.608458 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6lh5g\"" Jul 07 06:59:40.614801 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.614752 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-j2872"] Jul 07 06:59:40.618174 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.617144 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx"] Jul 07 06:59:40.619191 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.618422 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 06:59:40.619423 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.619350 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-ndp4z"] Jul 07 06:59:40.683999 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.683962 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:40.697496 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.697471 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mlgq\" (UniqueName: \"kubernetes.io/projected/b490dc3b-4f35-4db0-be34-cc804b50c62f-kube-api-access-5mlgq\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.697625 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.697519 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcb77\" (UniqueName: \"kubernetes.io/projected/08be047a-e24d-4b3a-868f-45d487a6c066-kube-api-access-qcb77\") pod \"console-operator-68b757fb67-ndp4z\" (UID: \"08be047a-e24d-4b3a-868f-45d487a6c066\") " pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 06:59:40.697625 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.697540 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b490dc3b-4f35-4db0-be34-cc804b50c62f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.697625 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.697582 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/08be047a-e24d-4b3a-868f-45d487a6c066-serving-cert\") pod \"console-operator-68b757fb67-ndp4z\" (UID: \"08be047a-e24d-4b3a-868f-45d487a6c066\") " pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 06:59:40.697625 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.697609 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:40.697839 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.697690 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b490dc3b-4f35-4db0-be34-cc804b50c62f-tmp\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.697839 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.697715 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/08be047a-e24d-4b3a-868f-45d487a6c066-config\") pod \"console-operator-68b757fb67-ndp4z\" (UID: \"08be047a-e24d-4b3a-868f-45d487a6c066\") " pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 06:59:40.697839 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.697737 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a63477af-964c-4ef4-a12c-fa205ac66c9d-tmp-dir\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:40.697839 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.697762 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b490dc3b-4f35-4db0-be34-cc804b50c62f-snapshots\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.697839 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.697787 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e8f7127-1872-4e43-a37d-d73d7b235a58-cert\") pod \"ingress-canary-dw7c2\" (UID: \"9e8f7127-1872-4e43-a37d-d73d7b235a58\") " pod="openshift-ingress-canary/ingress-canary-dw7c2" Jul 07 06:59:40.698079 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.697841 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/08be047a-e24d-4b3a-868f-45d487a6c066-trusted-ca\") pod \"console-operator-68b757fb67-ndp4z\" (UID: \"08be047a-e24d-4b3a-868f-45d487a6c066\") " pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 06:59:40.698079 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.697892 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b490dc3b-4f35-4db0-be34-cc804b50c62f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.698079 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.697915 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js28p\" (UniqueName: \"kubernetes.io/projected/a63477af-964c-4ef4-a12c-fa205ac66c9d-kube-api-access-js28p\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:40.698079 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.697937 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b490dc3b-4f35-4db0-be34-cc804b50c62f-serving-cert\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.698079 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.697955 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a63477af-964c-4ef4-a12c-fa205ac66c9d-config-volume\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:40.698079 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.698009 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j84g7\" (UniqueName: \"kubernetes.io/projected/9e8f7127-1872-4e43-a37d-d73d7b235a58-kube-api-access-j84g7\") pod \"ingress-canary-dw7c2\" (UID: \"9e8f7127-1872-4e43-a37d-d73d7b235a58\") " pod="openshift-ingress-canary/ingress-canary-dw7c2" Jul 07 06:59:40.788832 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.788757 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:40.798785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.798757 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b490dc3b-4f35-4db0-be34-cc804b50c62f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.798785 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.798786 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/08be047a-e24d-4b3a-868f-45d487a6c066-serving-cert\") pod \"console-operator-68b757fb67-ndp4z\" (UID: \"08be047a-e24d-4b3a-868f-45d487a6c066\") " pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 06:59:40.798965 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.798806 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:40.798965 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.798855 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/27f2c73a-c551-4fe0-9563-00c9cd765437-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-2xsrx\" (UID: \"27f2c73a-c551-4fe0-9563-00c9cd765437\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 06:59:40.798965 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.798886 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-2xsrx\" (UID: \"27f2c73a-c551-4fe0-9563-00c9cd765437\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 06:59:40.798965 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:40.798912 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 06:59:40.798965 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.798919 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b490dc3b-4f35-4db0-be34-cc804b50c62f-tmp\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.799183 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.798968 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/08be047a-e24d-4b3a-868f-45d487a6c066-config\") pod \"console-operator-68b757fb67-ndp4z\" (UID: \"08be047a-e24d-4b3a-868f-45d487a6c066\") " pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 06:59:40.799183 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:40.798979 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls podName:a63477af-964c-4ef4-a12c-fa205ac66c9d nodeName:}" failed. No retries permitted until 2026-07-07 06:59:41.298960467 +0000 UTC m=+36.634926782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls") pod "dns-default-pwjq8" (UID: "a63477af-964c-4ef4-a12c-fa205ac66c9d") : secret "dns-default-metrics-tls" not found Jul 07 06:59:40.799183 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.799017 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a63477af-964c-4ef4-a12c-fa205ac66c9d-tmp-dir\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:40.799183 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.799059 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b490dc3b-4f35-4db0-be34-cc804b50c62f-snapshots\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.799183 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.799085 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e8f7127-1872-4e43-a37d-d73d7b235a58-cert\") pod \"ingress-canary-dw7c2\" (UID: \"9e8f7127-1872-4e43-a37d-d73d7b235a58\") " pod="openshift-ingress-canary/ingress-canary-dw7c2" Jul 07 06:59:40.799183 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.799128 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/08be047a-e24d-4b3a-868f-45d487a6c066-trusted-ca\") pod \"console-operator-68b757fb67-ndp4z\" (UID: \"08be047a-e24d-4b3a-868f-45d487a6c066\") " pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 06:59:40.799459 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.799184 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b490dc3b-4f35-4db0-be34-cc804b50c62f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.799459 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.799211 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js28p\" (UniqueName: \"kubernetes.io/projected/a63477af-964c-4ef4-a12c-fa205ac66c9d-kube-api-access-js28p\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:40.799459 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.799243 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b490dc3b-4f35-4db0-be34-cc804b50c62f-serving-cert\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.799459 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.799268 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a63477af-964c-4ef4-a12c-fa205ac66c9d-config-volume\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:40.799459 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.799294 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j84g7\" (UniqueName: \"kubernetes.io/projected/9e8f7127-1872-4e43-a37d-d73d7b235a58-kube-api-access-j84g7\") pod \"ingress-canary-dw7c2\" (UID: \"9e8f7127-1872-4e43-a37d-d73d7b235a58\") " pod="openshift-ingress-canary/ingress-canary-dw7c2" Jul 07 06:59:40.799459 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.799318 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zfj5\" (UniqueName: \"kubernetes.io/projected/27f2c73a-c551-4fe0-9563-00c9cd765437-kube-api-access-4zfj5\") pod \"cluster-monitoring-operator-64786d9cbb-2xsrx\" (UID: \"27f2c73a-c551-4fe0-9563-00c9cd765437\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 06:59:40.799459 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.799363 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mlgq\" (UniqueName: \"kubernetes.io/projected/b490dc3b-4f35-4db0-be34-cc804b50c62f-kube-api-access-5mlgq\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.799459 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.799403 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcb77\" (UniqueName: \"kubernetes.io/projected/08be047a-e24d-4b3a-868f-45d487a6c066-kube-api-access-qcb77\") pod \"console-operator-68b757fb67-ndp4z\" (UID: \"08be047a-e24d-4b3a-868f-45d487a6c066\") " pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 06:59:40.799823 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.799501 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a63477af-964c-4ef4-a12c-fa205ac66c9d-tmp-dir\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:40.799878 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.799849 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b490dc3b-4f35-4db0-be34-cc804b50c62f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.799878 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.799852 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b490dc3b-4f35-4db0-be34-cc804b50c62f-tmp\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.800176 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.800142 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/08be047a-e24d-4b3a-868f-45d487a6c066-config\") pod \"console-operator-68b757fb67-ndp4z\" (UID: \"08be047a-e24d-4b3a-868f-45d487a6c066\") " pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 06:59:40.800292 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.800277 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b490dc3b-4f35-4db0-be34-cc804b50c62f-snapshots\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.800597 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.800571 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a63477af-964c-4ef4-a12c-fa205ac66c9d-config-volume\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:40.800982 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.800959 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/08be047a-e24d-4b3a-868f-45d487a6c066-trusted-ca\") pod \"console-operator-68b757fb67-ndp4z\" (UID: \"08be047a-e24d-4b3a-868f-45d487a6c066\") " pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 06:59:40.801923 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.801889 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b490dc3b-4f35-4db0-be34-cc804b50c62f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.804262 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.804202 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/08be047a-e24d-4b3a-868f-45d487a6c066-serving-cert\") pod \"console-operator-68b757fb67-ndp4z\" (UID: \"08be047a-e24d-4b3a-868f-45d487a6c066\") " pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 06:59:40.804262 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.804224 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b490dc3b-4f35-4db0-be34-cc804b50c62f-serving-cert\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.804409 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.804267 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e8f7127-1872-4e43-a37d-d73d7b235a58-cert\") pod \"ingress-canary-dw7c2\" (UID: \"9e8f7127-1872-4e43-a37d-d73d7b235a58\") " pod="openshift-ingress-canary/ingress-canary-dw7c2" Jul 07 06:59:40.810664 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.810638 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mlgq\" (UniqueName: \"kubernetes.io/projected/b490dc3b-4f35-4db0-be34-cc804b50c62f-kube-api-access-5mlgq\") pod \"insights-operator-5c847dcfc4-j2872\" (UID: \"b490dc3b-4f35-4db0-be34-cc804b50c62f\") " pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.810933 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.810902 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j84g7\" (UniqueName: \"kubernetes.io/projected/9e8f7127-1872-4e43-a37d-d73d7b235a58-kube-api-access-j84g7\") pod \"ingress-canary-dw7c2\" (UID: \"9e8f7127-1872-4e43-a37d-d73d7b235a58\") " pod="openshift-ingress-canary/ingress-canary-dw7c2" Jul 07 06:59:40.811179 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.811105 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcb77\" (UniqueName: \"kubernetes.io/projected/08be047a-e24d-4b3a-868f-45d487a6c066-kube-api-access-qcb77\") pod \"console-operator-68b757fb67-ndp4z\" (UID: \"08be047a-e24d-4b3a-868f-45d487a6c066\") " pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 06:59:40.811554 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.811532 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-js28p\" (UniqueName: \"kubernetes.io/projected/a63477af-964c-4ef4-a12c-fa205ac66c9d-kube-api-access-js28p\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:40.881459 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.881409 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:40.892442 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.892411 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dw7c2" Jul 07 06:59:40.900032 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.900004 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zfj5\" (UniqueName: \"kubernetes.io/projected/27f2c73a-c551-4fe0-9563-00c9cd765437-kube-api-access-4zfj5\") pod \"cluster-monitoring-operator-64786d9cbb-2xsrx\" (UID: \"27f2c73a-c551-4fe0-9563-00c9cd765437\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 06:59:40.900118 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.900072 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/27f2c73a-c551-4fe0-9563-00c9cd765437-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-2xsrx\" (UID: \"27f2c73a-c551-4fe0-9563-00c9cd765437\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 06:59:40.900118 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.900089 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-2xsrx\" (UID: \"27f2c73a-c551-4fe0-9563-00c9cd765437\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 06:59:40.900227 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:40.900210 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 06:59:40.900295 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:40.900285 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls podName:27f2c73a-c551-4fe0-9563-00c9cd765437 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:41.400271323 +0000 UTC m=+36.736237619 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-2xsrx" (UID: "27f2c73a-c551-4fe0-9563-00c9cd765437") : secret "cluster-monitoring-operator-tls" not found Jul 07 06:59:40.900868 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.900844 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/27f2c73a-c551-4fe0-9563-00c9cd765437-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-2xsrx\" (UID: \"27f2c73a-c551-4fe0-9563-00c9cd765437\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 06:59:40.907976 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.907953 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-j2872" Jul 07 06:59:40.910289 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.910267 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zfj5\" (UniqueName: \"kubernetes.io/projected/27f2c73a-c551-4fe0-9563-00c9cd765437-kube-api-access-4zfj5\") pod \"cluster-monitoring-operator-64786d9cbb-2xsrx\" (UID: \"27f2c73a-c551-4fe0-9563-00c9cd765437\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 06:59:40.920225 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:40.920195 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 06:59:41.101945 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:41.101859 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 06:59:41.105236 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:41.105214 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 06:59:41.304148 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:41.304118 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:41.304341 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:41.304276 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 06:59:41.304394 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:41.304370 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls podName:a63477af-964c-4ef4-a12c-fa205ac66c9d nodeName:}" failed. No retries permitted until 2026-07-07 06:59:42.304348656 +0000 UTC m=+37.640314951 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls") pod "dns-default-pwjq8" (UID: "a63477af-964c-4ef4-a12c-fa205ac66c9d") : secret "dns-default-metrics-tls" not found Jul 07 06:59:41.404695 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:41.404603 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-2xsrx\" (UID: \"27f2c73a-c551-4fe0-9563-00c9cd765437\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 06:59:41.404857 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:41.404769 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 06:59:41.404857 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:41.404850 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls podName:27f2c73a-c551-4fe0-9563-00c9cd765437 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:42.404828045 +0000 UTC m=+37.740794362 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-2xsrx" (UID: "27f2c73a-c551-4fe0-9563-00c9cd765437") : secret "cluster-monitoring-operator-tls" not found Jul 07 06:59:41.974699 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:41.974503 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-j2872"] Jul 07 06:59:41.977115 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:41.977052 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-ndp4z"] Jul 07 06:59:41.977505 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:41.977475 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb490dc3b_4f35_4db0_be34_cc804b50c62f.slice/crio-970c33bf542942f41eaa3bd7b05ef864481dd2c162993f7f5370741ec85d0aff WatchSource:0}: Error finding container 970c33bf542942f41eaa3bd7b05ef864481dd2c162993f7f5370741ec85d0aff: Status 404 returned error can't find the container with id 970c33bf542942f41eaa3bd7b05ef864481dd2c162993f7f5370741ec85d0aff Jul 07 06:59:41.982204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:41.981685 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dw7c2"] Jul 07 06:59:42.004851 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:42.004822 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e8f7127_1872_4e43_a37d_d73d7b235a58.slice/crio-f314ffdcbfaa7a1ebb0d46e5314763b57cdb95eb1ce3020bd73dce9be598cc0d WatchSource:0}: Error finding container f314ffdcbfaa7a1ebb0d46e5314763b57cdb95eb1ce3020bd73dce9be598cc0d: Status 404 returned error can't find the container with id f314ffdcbfaa7a1ebb0d46e5314763b57cdb95eb1ce3020bd73dce9be598cc0d Jul 07 06:59:42.102211 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:42.102191 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 06:59:42.102301 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:42.102191 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 06:59:42.104988 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:42.104891 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 06:59:42.105088 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:42.105008 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2lkzt\"" Jul 07 06:59:42.105088 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:42.105050 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 06:59:42.105088 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:42.105062 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-59nzl\"" Jul 07 06:59:42.105239 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:42.105052 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 06:59:42.312141 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:42.312116 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:42.312466 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:42.312273 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 06:59:42.312466 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:42.312348 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls podName:a63477af-964c-4ef4-a12c-fa205ac66c9d nodeName:}" failed. No retries permitted until 2026-07-07 06:59:44.312318693 +0000 UTC m=+39.648285006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls") pod "dns-default-pwjq8" (UID: "a63477af-964c-4ef4-a12c-fa205ac66c9d") : secret "dns-default-metrics-tls" not found Jul 07 06:59:42.407414 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:42.407354 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" event={"ID":"08be047a-e24d-4b3a-868f-45d487a6c066","Type":"ContainerStarted","Data":"6ae485855464497da19e73fafcbc4a5a524e2551feabb5560f77ca8bf9369688"} Jul 07 06:59:42.408378 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:42.408350 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dw7c2" event={"ID":"9e8f7127-1872-4e43-a37d-d73d7b235a58","Type":"ContainerStarted","Data":"f314ffdcbfaa7a1ebb0d46e5314763b57cdb95eb1ce3020bd73dce9be598cc0d"} Jul 07 06:59:42.410909 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:42.410886 2573 generic.go:356] "Generic (PLEG): container finished" podID="5bbe0aa3-493c-4233-b2a8-7feac9861c6f" containerID="a5c982e3404666493fc9d013ea7b5bc3177f0cc65c4fe66640460fb6c4a490eb" exitCode=0 Jul 07 06:59:42.410997 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:42.410917 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j9krq" event={"ID":"5bbe0aa3-493c-4233-b2a8-7feac9861c6f","Type":"ContainerDied","Data":"a5c982e3404666493fc9d013ea7b5bc3177f0cc65c4fe66640460fb6c4a490eb"} Jul 07 06:59:42.412015 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:42.411992 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-j2872" event={"ID":"b490dc3b-4f35-4db0-be34-cc804b50c62f","Type":"ContainerStarted","Data":"970c33bf542942f41eaa3bd7b05ef864481dd2c162993f7f5370741ec85d0aff"} Jul 07 06:59:42.413272 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:42.413251 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-2xsrx\" (UID: \"27f2c73a-c551-4fe0-9563-00c9cd765437\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 06:59:42.413415 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:42.413400 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 06:59:42.413492 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:42.413471 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls podName:27f2c73a-c551-4fe0-9563-00c9cd765437 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:44.413451835 +0000 UTC m=+39.749418133 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-2xsrx" (UID: "27f2c73a-c551-4fe0-9563-00c9cd765437") : secret "cluster-monitoring-operator-tls" not found Jul 07 06:59:42.563115 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:42.563085 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:42.663588 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:42.663225 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:43.387068 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:43.387032 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:43.419900 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:43.419862 2573 generic.go:356] "Generic (PLEG): container finished" podID="5bbe0aa3-493c-4233-b2a8-7feac9861c6f" containerID="9938ef5d86181e3ff0e998bcc817d61f05ab4924acdeb5975a7bd5cf0b95c06e" exitCode=0 Jul 07 06:59:43.420094 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:43.419945 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j9krq" event={"ID":"5bbe0aa3-493c-4233-b2a8-7feac9861c6f","Type":"ContainerDied","Data":"9938ef5d86181e3ff0e998bcc817d61f05ab4924acdeb5975a7bd5cf0b95c06e"} Jul 07 06:59:43.456682 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:43.456659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:43.984982 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:43.984790 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw"] Jul 07 06:59:44.000232 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:44.000210 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw"] Jul 07 06:59:44.000394 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:44.000300 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" Jul 07 06:59:44.003106 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:44.003085 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8kb8z\"" Jul 07 06:59:44.003245 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:44.003110 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 07 06:59:44.003245 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:44.003085 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 07 06:59:44.026966 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:44.026931 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4g8xw\" (UID: \"19033e5d-d309-40af-a5dd-99322d4c8bd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" Jul 07 06:59:44.027089 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:44.027057 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/19033e5d-d309-40af-a5dd-99322d4c8bd3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-4g8xw\" (UID: \"19033e5d-d309-40af-a5dd-99322d4c8bd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" Jul 07 06:59:44.127824 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:44.127789 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/19033e5d-d309-40af-a5dd-99322d4c8bd3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-4g8xw\" (UID: \"19033e5d-d309-40af-a5dd-99322d4c8bd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" Jul 07 06:59:44.127999 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:44.127886 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4g8xw\" (UID: \"19033e5d-d309-40af-a5dd-99322d4c8bd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" Jul 07 06:59:44.128074 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:44.128019 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 06:59:44.128126 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:44.128102 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert podName:19033e5d-d309-40af-a5dd-99322d4c8bd3 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:44.628083476 +0000 UTC m=+39.964049791 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-4g8xw" (UID: "19033e5d-d309-40af-a5dd-99322d4c8bd3") : secret "networking-console-plugin-cert" not found Jul 07 06:59:44.128499 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:44.128481 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/19033e5d-d309-40af-a5dd-99322d4c8bd3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-4g8xw\" (UID: \"19033e5d-d309-40af-a5dd-99322d4c8bd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" Jul 07 06:59:44.329743 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:44.329708 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:44.329927 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:44.329845 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 06:59:44.329927 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:44.329918 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls podName:a63477af-964c-4ef4-a12c-fa205ac66c9d nodeName:}" failed. No retries permitted until 2026-07-07 06:59:48.329902955 +0000 UTC m=+43.665869266 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls") pod "dns-default-pwjq8" (UID: "a63477af-964c-4ef4-a12c-fa205ac66c9d") : secret "dns-default-metrics-tls" not found Jul 07 06:59:44.426731 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:44.426699 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j9krq" event={"ID":"5bbe0aa3-493c-4233-b2a8-7feac9861c6f","Type":"ContainerStarted","Data":"10117f5950f734bfeadd5926d96f56834aae8f034ce4552af62360e57851e658"} Jul 07 06:59:44.430941 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:44.430914 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-2xsrx\" (UID: \"27f2c73a-c551-4fe0-9563-00c9cd765437\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 06:59:44.431128 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:44.431101 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 06:59:44.431243 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:44.431166 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls podName:27f2c73a-c551-4fe0-9563-00c9cd765437 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:48.431147173 +0000 UTC m=+43.767113473 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-2xsrx" (UID: "27f2c73a-c551-4fe0-9563-00c9cd765437") : secret "cluster-monitoring-operator-tls" not found Jul 07 06:59:44.451859 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:44.451810 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j9krq" podStartSLOduration=6.705890294 podStartE2EDuration="38.451795677s" totalImagesPullingTime="31.745905383s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:59:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:44.450280148 +0000 UTC m=+39.786246466" watchObservedRunningTime="2026-07-07 06:59:44.451795677 +0000 UTC m=+39.787761995" Jul 07 06:59:44.633387 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:44.633306 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4g8xw\" (UID: \"19033e5d-d309-40af-a5dd-99322d4c8bd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" Jul 07 06:59:44.633514 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:44.633454 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 06:59:44.633551 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:44.633515 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert podName:19033e5d-d309-40af-a5dd-99322d4c8bd3 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:45.633500295 +0000 UTC m=+40.969466609 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-4g8xw" (UID: "19033e5d-d309-40af-a5dd-99322d4c8bd3") : secret "networking-console-plugin-cert" not found Jul 07 06:59:45.144428 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:45.144385 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:45.214360 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:45.214307 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:45.531483 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:45.531448 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:45.597387 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:45.597348 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:45.641854 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:45.641823 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4g8xw\" (UID: \"19033e5d-d309-40af-a5dd-99322d4c8bd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" Jul 07 06:59:45.641994 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:45.641962 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 06:59:45.642057 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:45.642046 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert podName:19033e5d-d309-40af-a5dd-99322d4c8bd3 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:47.642031212 +0000 UTC m=+42.977997511 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-4g8xw" (UID: "19033e5d-d309-40af-a5dd-99322d4c8bd3") : secret "networking-console-plugin-cert" not found Jul 07 06:59:45.831483 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:45.831454 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:45.936294 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:45.936211 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:46.224479 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.224395 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:46.250773 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.250749 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-2455j"] Jul 07 06:59:46.273781 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.273758 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-2455j"] Jul 07 06:59:46.273897 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.273850 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2455j" Jul 07 06:59:46.276348 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.276310 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ph7l9\"" Jul 07 06:59:46.276436 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.276354 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 07 06:59:46.277081 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.277065 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 07 06:59:46.313177 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.313154 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:46.346767 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.346744 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpnbk\" (UniqueName: \"kubernetes.io/projected/978b2765-a258-4b3b-ab4b-0f0e5188b876-kube-api-access-rpnbk\") pod \"migrator-d57d58cc9-2455j\" (UID: \"978b2765-a258-4b3b-ab4b-0f0e5188b876\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2455j" Jul 07 06:59:46.413568 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.413541 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:46.433595 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.433561 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-j2872" event={"ID":"b490dc3b-4f35-4db0-be34-cc804b50c62f","Type":"ContainerStarted","Data":"9d2eba1008199829345d98169271bdee7cc0c7e9020661ec3040663ad530eb75"} Jul 07 06:59:46.435053 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.435033 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/0.log" Jul 07 06:59:46.435153 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.435070 2573 generic.go:356] "Generic (PLEG): container finished" podID="08be047a-e24d-4b3a-868f-45d487a6c066" containerID="8f2a66348e2625f82bb15653cde2a4f45091b03f06249d3def69aef486cdbe8c" exitCode=255 Jul 07 06:59:46.435214 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.435153 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" event={"ID":"08be047a-e24d-4b3a-868f-45d487a6c066","Type":"ContainerDied","Data":"8f2a66348e2625f82bb15653cde2a4f45091b03f06249d3def69aef486cdbe8c"} Jul 07 06:59:46.435312 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.435297 2573 scope.go:117] "RemoveContainer" containerID="8f2a66348e2625f82bb15653cde2a4f45091b03f06249d3def69aef486cdbe8c" Jul 07 06:59:46.436636 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.436617 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dw7c2" event={"ID":"9e8f7127-1872-4e43-a37d-d73d7b235a58","Type":"ContainerStarted","Data":"c7e2dbed11a7935c84156e0bac260cf61581a9330249b4d3fce086b2d13f0665"} Jul 07 06:59:46.447550 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.447524 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpnbk\" (UniqueName: \"kubernetes.io/projected/978b2765-a258-4b3b-ab4b-0f0e5188b876-kube-api-access-rpnbk\") pod \"migrator-d57d58cc9-2455j\" (UID: \"978b2765-a258-4b3b-ab4b-0f0e5188b876\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2455j" Jul 07 06:59:46.451647 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.451607 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-j2872" podStartSLOduration=2.627028867 podStartE2EDuration="6.451595522s" totalImagesPullingTime="3.824566655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:46.451245109 +0000 UTC m=+41.787211426" watchObservedRunningTime="2026-07-07 06:59:46.451595522 +0000 UTC m=+41.787561817" Jul 07 06:59:46.459037 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.459018 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpnbk\" (UniqueName: \"kubernetes.io/projected/978b2765-a258-4b3b-ab4b-0f0e5188b876-kube-api-access-rpnbk\") pod \"migrator-d57d58cc9-2455j\" (UID: \"978b2765-a258-4b3b-ab4b-0f0e5188b876\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2455j" Jul 07 06:59:46.469168 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.469131 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dw7c2" podStartSLOduration=2.640642055 podStartE2EDuration="6.469118335s" totalImagesPullingTime="3.82847628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:46.467901754 +0000 UTC m=+41.803868071" watchObservedRunningTime="2026-07-07 06:59:46.469118335 +0000 UTC m=+41.805084651" Jul 07 06:59:46.582925 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.582816 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2455j" Jul 07 06:59:46.708831 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.708806 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:46.725595 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.725567 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-2455j"] Jul 07 06:59:46.729550 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:46.729523 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod978b2765_a258_4b3b_ab4b_0f0e5188b876.slice/crio-9f0d0fb9569a6cdaf1f899185e37af133e0b6e9cae318e45a179e89d828c674b WatchSource:0}: Error finding container 9f0d0fb9569a6cdaf1f899185e37af133e0b6e9cae318e45a179e89d828c674b: Status 404 returned error can't find the container with id 9f0d0fb9569a6cdaf1f899185e37af133e0b6e9cae318e45a179e89d828c674b Jul 07 06:59:46.808729 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.808707 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:46.975977 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:46.975945 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:47.076459 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.076373 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:47.439959 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.439884 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2455j" event={"ID":"978b2765-a258-4b3b-ab4b-0f0e5188b876","Type":"ContainerStarted","Data":"9f0d0fb9569a6cdaf1f899185e37af133e0b6e9cae318e45a179e89d828c674b"} Jul 07 06:59:47.441617 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.441592 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 06:59:47.442041 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.442026 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/0.log" Jul 07 06:59:47.442115 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.442061 2573 generic.go:356] "Generic (PLEG): container finished" podID="08be047a-e24d-4b3a-868f-45d487a6c066" containerID="4de4b466a9885ef6ed515987f135f1bda30ade8c69f2feb17ebf3cf0cbd95abb" exitCode=255 Jul 07 06:59:47.442115 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.442089 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" event={"ID":"08be047a-e24d-4b3a-868f-45d487a6c066","Type":"ContainerDied","Data":"4de4b466a9885ef6ed515987f135f1bda30ade8c69f2feb17ebf3cf0cbd95abb"} Jul 07 06:59:47.442192 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.442156 2573 scope.go:117] "RemoveContainer" containerID="8f2a66348e2625f82bb15653cde2a4f45091b03f06249d3def69aef486cdbe8c" Jul 07 06:59:47.442541 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.442516 2573 scope.go:117] "RemoveContainer" containerID="4de4b466a9885ef6ed515987f135f1bda30ade8c69f2feb17ebf3cf0cbd95abb" Jul 07 06:59:47.442719 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:47.442697 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-ndp4z_openshift-console-operator(08be047a-e24d-4b3a-868f-45d487a6c066)\"" pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" podUID="08be047a-e24d-4b3a-868f-45d487a6c066" Jul 07 06:59:47.444460 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.444442 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9wst6"] Jul 07 06:59:47.470088 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.470069 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9wst6"] Jul 07 06:59:47.470184 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.470165 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:47.473865 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.473842 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 06:59:47.473865 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.473847 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s62sg\"" Jul 07 06:59:47.474941 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.474923 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 06:59:47.486984 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.486962 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:47.551364 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.551269 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:47.556154 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.556130 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s9ng\" (UniqueName: \"kubernetes.io/projected/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-api-access-8s9ng\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:47.556284 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.556166 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9990b17f-cccd-4d39-a280-f179cf67e9c9-data-volume\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:47.556453 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.556426 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:47.556683 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.556665 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:47.556814 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.556789 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9990b17f-cccd-4d39-a280-f179cf67e9c9-crio-socket\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:47.657293 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.657260 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:47.657648 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.657309 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:47.657648 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.657357 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9990b17f-cccd-4d39-a280-f179cf67e9c9-crio-socket\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:47.657648 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.657379 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s9ng\" (UniqueName: \"kubernetes.io/projected/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-api-access-8s9ng\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:47.657648 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.657394 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9990b17f-cccd-4d39-a280-f179cf67e9c9-data-volume\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:47.657648 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:47.657412 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 06:59:47.657648 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:47.657465 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 06:59:47.657648 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.657498 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4g8xw\" (UID: \"19033e5d-d309-40af-a5dd-99322d4c8bd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" Jul 07 06:59:47.657648 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:47.657510 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls podName:9990b17f-cccd-4d39-a280-f179cf67e9c9 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:48.157491365 +0000 UTC m=+43.493457673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9wst6" (UID: "9990b17f-cccd-4d39-a280-f179cf67e9c9") : secret "insights-runtime-extractor-tls" not found Jul 07 06:59:47.657648 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:47.657539 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm podName:9990b17f-cccd-4d39-a280-f179cf67e9c9 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:48.157526916 +0000 UTC m=+43.493493225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9wst6" (UID: "9990b17f-cccd-4d39-a280-f179cf67e9c9") : configmap "kube-rbac-proxy" not found Jul 07 06:59:47.657648 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:47.657569 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 06:59:47.657648 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.657577 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9990b17f-cccd-4d39-a280-f179cf67e9c9-crio-socket\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:47.657648 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:47.657613 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert podName:19033e5d-d309-40af-a5dd-99322d4c8bd3 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:51.657595499 +0000 UTC m=+46.993561807 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-4g8xw" (UID: "19033e5d-d309-40af-a5dd-99322d4c8bd3") : secret "networking-console-plugin-cert" not found Jul 07 06:59:47.657996 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.657686 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9990b17f-cccd-4d39-a280-f179cf67e9c9-data-volume\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:47.667027 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:47.667008 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s9ng\" (UniqueName: \"kubernetes.io/projected/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-api-access-8s9ng\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:48.162731 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.162694 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:48.162900 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.162745 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:48.162900 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:48.162810 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 06:59:48.162900 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:48.162846 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 06:59:48.162900 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:48.162855 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm podName:9990b17f-cccd-4d39-a280-f179cf67e9c9 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:49.16284274 +0000 UTC m=+44.498809035 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9wst6" (UID: "9990b17f-cccd-4d39-a280-f179cf67e9c9") : configmap "kube-rbac-proxy" not found Jul 07 06:59:48.163082 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:48.162954 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls podName:9990b17f-cccd-4d39-a280-f179cf67e9c9 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:49.162943836 +0000 UTC m=+44.498910130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9wst6" (UID: "9990b17f-cccd-4d39-a280-f179cf67e9c9") : secret "insights-runtime-extractor-tls" not found Jul 07 06:59:48.364442 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.364409 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:48.364567 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:48.364552 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 06:59:48.364615 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:48.364606 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls podName:a63477af-964c-4ef4-a12c-fa205ac66c9d nodeName:}" failed. No retries permitted until 2026-07-07 06:59:56.364592226 +0000 UTC m=+51.700558524 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls") pod "dns-default-pwjq8" (UID: "a63477af-964c-4ef4-a12c-fa205ac66c9d") : secret "dns-default-metrics-tls" not found Jul 07 06:59:48.448119 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.448090 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2455j" event={"ID":"978b2765-a258-4b3b-ab4b-0f0e5188b876","Type":"ContainerStarted","Data":"2af44af84336ca540939288d8624813e2498957e3f7eea89f389df524c9b1854"} Jul 07 06:59:48.449437 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.449421 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 06:59:48.449715 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.449701 2573 scope.go:117] "RemoveContainer" containerID="4de4b466a9885ef6ed515987f135f1bda30ade8c69f2feb17ebf3cf0cbd95abb" Jul 07 06:59:48.449846 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:48.449829 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-ndp4z_openshift-console-operator(08be047a-e24d-4b3a-868f-45d487a6c066)\"" pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" podUID="08be047a-e24d-4b3a-868f-45d487a6c066" Jul 07 06:59:48.465338 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.465298 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-2xsrx\" (UID: \"27f2c73a-c551-4fe0-9563-00c9cd765437\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 06:59:48.465451 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:48.465435 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 06:59:48.465503 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:48.465491 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls podName:27f2c73a-c551-4fe0-9563-00c9cd765437 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:56.465474586 +0000 UTC m=+51.801440881 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-2xsrx" (UID: "27f2c73a-c551-4fe0-9563-00c9cd765437") : secret "cluster-monitoring-operator-tls" not found Jul 07 06:59:48.591151 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.591127 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-vr649"] Jul 07 06:59:48.607002 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.606982 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-vr649" Jul 07 06:59:48.609497 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.609471 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-vr649"] Jul 07 06:59:48.609951 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.609928 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 07 06:59:48.610030 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.609936 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-mzc75\"" Jul 07 06:59:48.610463 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.610450 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 07 06:59:48.610678 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.610659 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 07 06:59:48.610750 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.610735 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 07 06:59:48.638764 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.638740 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:48.668031 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.667987 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e2c11643-d735-4eb2-9e90-aaa88761ebeb-signing-cabundle\") pod \"service-ca-758f599c5-vr649\" (UID: \"e2c11643-d735-4eb2-9e90-aaa88761ebeb\") " pod="openshift-service-ca/service-ca-758f599c5-vr649" Jul 07 06:59:48.668031 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.668026 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwks5\" (UniqueName: \"kubernetes.io/projected/e2c11643-d735-4eb2-9e90-aaa88761ebeb-kube-api-access-bwks5\") pod \"service-ca-758f599c5-vr649\" (UID: \"e2c11643-d735-4eb2-9e90-aaa88761ebeb\") " pod="openshift-service-ca/service-ca-758f599c5-vr649" Jul 07 06:59:48.668382 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.668047 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e2c11643-d735-4eb2-9e90-aaa88761ebeb-signing-key\") pod \"service-ca-758f599c5-vr649\" (UID: \"e2c11643-d735-4eb2-9e90-aaa88761ebeb\") " pod="openshift-service-ca/service-ca-758f599c5-vr649" Jul 07 06:59:48.739992 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.739966 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:48.769224 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.769202 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e2c11643-d735-4eb2-9e90-aaa88761ebeb-signing-cabundle\") pod \"service-ca-758f599c5-vr649\" (UID: \"e2c11643-d735-4eb2-9e90-aaa88761ebeb\") " pod="openshift-service-ca/service-ca-758f599c5-vr649" Jul 07 06:59:48.769286 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.769247 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwks5\" (UniqueName: \"kubernetes.io/projected/e2c11643-d735-4eb2-9e90-aaa88761ebeb-kube-api-access-bwks5\") pod \"service-ca-758f599c5-vr649\" (UID: \"e2c11643-d735-4eb2-9e90-aaa88761ebeb\") " pod="openshift-service-ca/service-ca-758f599c5-vr649" Jul 07 06:59:48.769286 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.769264 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e2c11643-d735-4eb2-9e90-aaa88761ebeb-signing-key\") pod \"service-ca-758f599c5-vr649\" (UID: \"e2c11643-d735-4eb2-9e90-aaa88761ebeb\") " pod="openshift-service-ca/service-ca-758f599c5-vr649" Jul 07 06:59:48.769853 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.769834 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e2c11643-d735-4eb2-9e90-aaa88761ebeb-signing-cabundle\") pod \"service-ca-758f599c5-vr649\" (UID: \"e2c11643-d735-4eb2-9e90-aaa88761ebeb\") " pod="openshift-service-ca/service-ca-758f599c5-vr649" Jul 07 06:59:48.771600 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.771581 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e2c11643-d735-4eb2-9e90-aaa88761ebeb-signing-key\") pod \"service-ca-758f599c5-vr649\" (UID: \"e2c11643-d735-4eb2-9e90-aaa88761ebeb\") " pod="openshift-service-ca/service-ca-758f599c5-vr649" Jul 07 06:59:48.780043 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.780026 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwks5\" (UniqueName: \"kubernetes.io/projected/e2c11643-d735-4eb2-9e90-aaa88761ebeb-kube-api-access-bwks5\") pod \"service-ca-758f599c5-vr649\" (UID: \"e2c11643-d735-4eb2-9e90-aaa88761ebeb\") " pod="openshift-service-ca/service-ca-758f599c5-vr649" Jul 07 06:59:48.917125 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:48.917098 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-vr649" Jul 07 06:59:49.058341 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:49.058295 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-vr649"] Jul 07 06:59:49.061141 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:49.061105 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2c11643_d735_4eb2_9e90_aaa88761ebeb.slice/crio-58d818d4a9ae7381b93caa43b06f5beeb1db0c02e797c0615f8796837e85eb85 WatchSource:0}: Error finding container 58d818d4a9ae7381b93caa43b06f5beeb1db0c02e797c0615f8796837e85eb85: Status 404 returned error can't find the container with id 58d818d4a9ae7381b93caa43b06f5beeb1db0c02e797c0615f8796837e85eb85 Jul 07 06:59:49.171668 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:49.171643 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:49.171788 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:49.171706 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:49.171847 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:49.171804 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 06:59:49.171847 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:49.171830 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 06:59:49.171945 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:49.171888 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls podName:9990b17f-cccd-4d39-a280-f179cf67e9c9 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:51.17186811 +0000 UTC m=+46.507834407 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9wst6" (UID: "9990b17f-cccd-4d39-a280-f179cf67e9c9") : secret "insights-runtime-extractor-tls" not found Jul 07 06:59:49.171945 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:49.171912 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm podName:9990b17f-cccd-4d39-a280-f179cf67e9c9 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:51.17190032 +0000 UTC m=+46.507866631 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9wst6" (UID: "9990b17f-cccd-4d39-a280-f179cf67e9c9") : configmap "kube-rbac-proxy" not found Jul 07 06:59:49.453406 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:49.453370 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-vr649" event={"ID":"e2c11643-d735-4eb2-9e90-aaa88761ebeb","Type":"ContainerStarted","Data":"58d818d4a9ae7381b93caa43b06f5beeb1db0c02e797c0615f8796837e85eb85"} Jul 07 06:59:49.454931 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:49.454906 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2455j" event={"ID":"978b2765-a258-4b3b-ab4b-0f0e5188b876","Type":"ContainerStarted","Data":"9c71486bf590ed857c086968c7c0ba5fff63638dceb16503722a3ab2bad3682b"} Jul 07 06:59:49.480183 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:49.480141 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2455j" podStartSLOduration=1.878871986 podStartE2EDuration="3.480127072s" totalImagesPullingTime="1.601255086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:59:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:49.479730429 +0000 UTC m=+44.815696746" watchObservedRunningTime="2026-07-07 06:59:49.480127072 +0000 UTC m=+44.816093438" Jul 07 06:59:49.588606 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:49.588579 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:49.697498 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:49.697463 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:50.022853 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:50.022825 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2g66m_a29da4ef-3d23-4022-bd07-dd84bae55b5e/dns-node-resolver/0.log" Jul 07 06:59:50.221471 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:50.221440 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:50.920482 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:50.920447 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 06:59:50.920832 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:50.920519 2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 06:59:50.920963 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:50.920947 2573 scope.go:117] "RemoveContainer" containerID="4de4b466a9885ef6ed515987f135f1bda30ade8c69f2feb17ebf3cf0cbd95abb" Jul 07 06:59:50.921151 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:50.921130 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-ndp4z_openshift-console-operator(08be047a-e24d-4b3a-868f-45d487a6c066)\"" pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" podUID="08be047a-e24d-4b3a-868f-45d487a6c066" Jul 07 06:59:51.023312 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:51.023287 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bm5dq_0c622a1f-4a29-45b7-a0de-785b67e27a5c/node-ca/0.log" Jul 07 06:59:51.189233 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:51.189157 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:51.189233 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:51.189205 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:51.189460 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:51.189344 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 06:59:51.189460 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:51.189369 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 06:59:51.189460 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:51.189414 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls podName:9990b17f-cccd-4d39-a280-f179cf67e9c9 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:55.189395588 +0000 UTC m=+50.525361882 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9wst6" (UID: "9990b17f-cccd-4d39-a280-f179cf67e9c9") : secret "insights-runtime-extractor-tls" not found Jul 07 06:59:51.189460 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:51.189434 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm podName:9990b17f-cccd-4d39-a280-f179cf67e9c9 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:55.189423851 +0000 UTC m=+50.525390152 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9wst6" (UID: "9990b17f-cccd-4d39-a280-f179cf67e9c9") : configmap "kube-rbac-proxy" not found Jul 07 06:59:51.694502 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:51.694470 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4g8xw\" (UID: \"19033e5d-d309-40af-a5dd-99322d4c8bd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" Jul 07 06:59:51.694673 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:51.694653 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 06:59:51.694742 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:51.694732 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert podName:19033e5d-d309-40af-a5dd-99322d4c8bd3 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:59.694713202 +0000 UTC m=+55.030679502 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-4g8xw" (UID: "19033e5d-d309-40af-a5dd-99322d4c8bd3") : secret "networking-console-plugin-cert" not found Jul 07 06:59:52.021503 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:52.021473 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dw7c2_9e8f7127-1872-4e43-a37d-d73d7b235a58/serve-healthcheck-canary/0.log" Jul 07 06:59:52.361609 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:52.361519 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:52.423609 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:52.423577 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-2455j_978b2765-a258-4b3b-ab4b-0f0e5188b876/migrator/0.log" Jul 07 06:59:52.465835 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:52.465776 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-vr649" event={"ID":"e2c11643-d735-4eb2-9e90-aaa88761ebeb","Type":"ContainerStarted","Data":"f4b544865f8e202c96dba35a75fdd63b37ddba2d639d8a11eedb5d5d41eb42a6"} Jul 07 06:59:52.485013 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:52.484966 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-vr649" podStartSLOduration=2.074876708 podStartE2EDuration="4.484947493s" totalImagesPullingTime="2.410070785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:59:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:52.484683559 +0000 UTC m=+47.820649876" watchObservedRunningTime="2026-07-07 06:59:52.484947493 +0000 UTC m=+47.820913812" Jul 07 06:59:52.577250 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:52.577215 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:52.627979 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:52.627916 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-2455j_978b2765-a258-4b3b-ab4b-0f0e5188b876/graceful-termination/0.log" Jul 07 06:59:52.702620 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:52.702590 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:52.812339 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:52.812304 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:52.892537 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:52.892454 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:53.060190 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:53.060157 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:53.310690 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:53.310660 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:54.256073 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:54.256043 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:54.459120 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:54.459090 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:54.657189 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:54.657115 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:54.854421 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:54.854383 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:55.049548 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:55.049509 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:55.220934 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:55.220907 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:55.221090 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:55.221006 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 06:59:55.221090 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:55.221036 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 06:59:55.221160 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:55.221095 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 06:59:55.221160 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:55.221107 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm podName:9990b17f-cccd-4d39-a280-f179cf67e9c9 nodeName:}" failed. No retries permitted until 2026-07-07 07:00:03.221088673 +0000 UTC m=+58.557054971 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9wst6" (UID: "9990b17f-cccd-4d39-a280-f179cf67e9c9") : configmap "kube-rbac-proxy" not found Jul 07 06:59:55.221160 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:55.221130 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls podName:9990b17f-cccd-4d39-a280-f179cf67e9c9 nodeName:}" failed. No retries permitted until 2026-07-07 07:00:03.221119336 +0000 UTC m=+58.557085631 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9wst6" (UID: "9990b17f-cccd-4d39-a280-f179cf67e9c9") : secret "insights-runtime-extractor-tls" not found Jul 07 06:59:55.770388 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:55.770349 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:56.066735 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:56.066658 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:56.157308 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:56.157271 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:56.431060 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:56.430985 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:56.433304 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:56.433285 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a63477af-964c-4ef4-a12c-fa205ac66c9d-metrics-tls\") pod \"dns-default-pwjq8\" (UID: \"a63477af-964c-4ef4-a12c-fa205ac66c9d\") " pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:56.461262 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:56.461234 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:56.484898 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:56.484876 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:56.531738 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:56.531704 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-2xsrx\" (UID: \"27f2c73a-c551-4fe0-9563-00c9cd765437\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 06:59:56.531928 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:56.531901 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 06:59:56.532015 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:56.532001 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls podName:27f2c73a-c551-4fe0-9563-00c9cd765437 nodeName:}" failed. No retries permitted until 2026-07-07 07:00:12.531975945 +0000 UTC m=+67.867942279 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-2xsrx" (UID: "27f2c73a-c551-4fe0-9563-00c9cd765437") : secret "cluster-monitoring-operator-tls" not found Jul 07 06:59:56.632061 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:56.632038 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pwjq8"] Jul 07 06:59:56.634273 ip-10-0-131-241 kubenswrapper[2573]: W0707 06:59:56.634244 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda63477af_964c_4ef4_a12c_fa205ac66c9d.slice/crio-bf8d0e23e884c191dae45da2f485511bf8da78f4420359d3d984584c3f60582b WatchSource:0}: Error finding container bf8d0e23e884c191dae45da2f485511bf8da78f4420359d3d984584c3f60582b: Status 404 returned error can't find the container with id bf8d0e23e884c191dae45da2f485511bf8da78f4420359d3d984584c3f60582b Jul 07 06:59:56.654411 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:56.654359 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:56.727490 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:56.727423 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:56.848936 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:56.848907 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:57.049541 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:57.049513 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:57.481948 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:57.481859 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pwjq8" event={"ID":"a63477af-964c-4ef4-a12c-fa205ac66c9d","Type":"ContainerStarted","Data":"bf8d0e23e884c191dae45da2f485511bf8da78f4420359d3d984584c3f60582b"} Jul 07 06:59:57.875257 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:57.875226 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:58.250437 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:58.250403 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:58.670055 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:58.669977 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:58.756434 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:58.756394 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:59.051919 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:59.051893 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:59.303268 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:59.303235 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:59.438727 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:59.438655 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:59.489987 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:59.489957 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pwjq8" event={"ID":"a63477af-964c-4ef4-a12c-fa205ac66c9d","Type":"ContainerStarted","Data":"ce0b615b6aaa6393b61dbdc96b025139917dcd5ab1605aa97fd0c4db5576eb89"} Jul 07 06:59:59.490122 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:59.490003 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pwjq8" event={"ID":"a63477af-964c-4ef4-a12c-fa205ac66c9d","Type":"ContainerStarted","Data":"a0a3e954f224bff6840d9dba1e572333aa074c5277f91af2cbdc697fefec34fe"} Jul 07 06:59:59.490122 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:59.490104 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pwjq8" Jul 07 06:59:59.510684 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:59.510638 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pwjq8" podStartSLOduration=17.216548157 podStartE2EDuration="19.510625843s" totalImagesPullingTime="2.294077686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:59.510423812 +0000 UTC m=+54.846390129" watchObservedRunningTime="2026-07-07 06:59:59.510625843 +0000 UTC m=+54.846592160" Jul 07 06:59:59.653233 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:59.653201 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:59.761204 ip-10-0-131-241 kubenswrapper[2573]: I0707 06:59:59.761174 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4g8xw\" (UID: \"19033e5d-d309-40af-a5dd-99322d4c8bd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" Jul 07 06:59:59.761382 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:59.761360 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 06:59:59.761454 ip-10-0-131-241 kubenswrapper[2573]: E0707 06:59:59.761444 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert podName:19033e5d-d309-40af-a5dd-99322d4c8bd3 nodeName:}" failed. No retries permitted until 2026-07-07 07:00:15.761421851 +0000 UTC m=+71.097388161 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-4g8xw" (UID: "19033e5d-d309-40af-a5dd-99322d4c8bd3") : secret "networking-console-plugin-cert" not found Jul 07 07:00:00.448168 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:00.448132 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:00.870876 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:00.870828 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:01.093628 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:01.093598 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:01.261580 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:01.261553 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:01.454787 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:01.454750 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:02.101026 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:02.100999 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:02.263581 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:02.263548 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:02.654026 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:02.653997 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:03.111086 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:03.111057 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:03.183231 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:03.183206 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:03.289876 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:03.289842 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 07:00:03.290020 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:03.289899 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 07:00:03.290388 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:03.290362 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9990b17f-cccd-4d39-a280-f179cf67e9c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 07:00:03.292075 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:03.292053 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9990b17f-cccd-4d39-a280-f179cf67e9c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9wst6\" (UID: \"9990b17f-cccd-4d39-a280-f179cf67e9c9\") " pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 07:00:03.379010 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:03.378925 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9wst6" Jul 07 07:00:03.502716 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:03.502682 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9wst6"] Jul 07 07:00:03.505996 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:00:03.505969 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9990b17f_cccd_4d39_a280_f179cf67e9c9.slice/crio-2da873374d8258540dc57872323fc17580127aa6f5a839c40c48f18b176feae9 WatchSource:0}: Error finding container 2da873374d8258540dc57872323fc17580127aa6f5a839c40c48f18b176feae9: Status 404 returned error can't find the container with id 2da873374d8258540dc57872323fc17580127aa6f5a839c40c48f18b176feae9 Jul 07 07:00:04.511973 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:04.511887 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9wst6" event={"ID":"9990b17f-cccd-4d39-a280-f179cf67e9c9","Type":"ContainerStarted","Data":"2da873374d8258540dc57872323fc17580127aa6f5a839c40c48f18b176feae9"} Jul 07 07:00:06.102512 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:06.102481 2573 scope.go:117] "RemoveContainer" containerID="4de4b466a9885ef6ed515987f135f1bda30ade8c69f2feb17ebf3cf0cbd95abb" Jul 07 07:00:06.406077 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:06.406014 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gfdjz" Jul 07 07:00:08.527729 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:08.527695 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9wst6" event={"ID":"9990b17f-cccd-4d39-a280-f179cf67e9c9","Type":"ContainerStarted","Data":"0c6b563fbcd63957f4b790e2904b163b04a2312637fd671f5e3ff076402dcbfc"} Jul 07 07:00:08.528117 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:08.527763 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9wst6" event={"ID":"9990b17f-cccd-4d39-a280-f179cf67e9c9","Type":"ContainerStarted","Data":"6936ecd8d23787d4bce86c8df67e20af18ef1e0148485a557845e9561961b4b5"} Jul 07 07:00:08.529411 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:08.529392 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:00:08.529508 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:08.529440 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" event={"ID":"08be047a-e24d-4b3a-868f-45d487a6c066","Type":"ContainerStarted","Data":"32a31f3816a8807fcf3e46fc5902899c7057eec7f47eab61aa31c89b2f53ad17"} Jul 07 07:00:08.529712 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:08.529697 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 07:00:08.534060 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:08.534041 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" Jul 07 07:00:08.553243 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:08.553205 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-ndp4z" podStartSLOduration=24.728108883 podStartE2EDuration="28.553192974s" totalImagesPullingTime="3.825084091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:00:08.552946107 +0000 UTC m=+63.888912425" watchObservedRunningTime="2026-07-07 07:00:08.553192974 +0000 UTC m=+63.889159290" Jul 07 07:00:08.665317 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:08.665281 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:08.771511 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:08.771481 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:08.986738 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:08.986656 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:09.337040 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.337007 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:09.450988 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.450951 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:09.495698 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.495671 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pwjq8" Jul 07 07:00:09.737580 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.737504 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v"] Jul 07 07:00:09.739795 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.739776 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.743000 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.742973 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 07:00:09.743369 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.743347 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 07 07:00:09.743940 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.743920 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 07 07:00:09.743940 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.743938 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 07 07:00:09.744103 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.744016 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 07 07:00:09.745022 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.744989 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 07:00:09.745022 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.745012 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 07:00:09.765075 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.765055 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v"] Jul 07 07:00:09.807734 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.807697 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:09.851317 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.851288 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ec09d40a-acdf-4b61-a322-a63fe33a75d4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.851490 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.851357 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6987j\" (UniqueName: \"kubernetes.io/projected/ec09d40a-acdf-4b61-a322-a63fe33a75d4-kube-api-access-6987j\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.851490 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.851400 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ec09d40a-acdf-4b61-a322-a63fe33a75d4-ca\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.851490 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.851416 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ec09d40a-acdf-4b61-a322-a63fe33a75d4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.851490 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.851454 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ec09d40a-acdf-4b61-a322-a63fe33a75d4-hub\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.851490 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.851477 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec09d40a-acdf-4b61-a322-a63fe33a75d4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.952301 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.952267 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ec09d40a-acdf-4b61-a322-a63fe33a75d4-hub\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.952498 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.952307 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec09d40a-acdf-4b61-a322-a63fe33a75d4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.952498 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.952393 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ec09d40a-acdf-4b61-a322-a63fe33a75d4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.952498 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.952447 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6987j\" (UniqueName: \"kubernetes.io/projected/ec09d40a-acdf-4b61-a322-a63fe33a75d4-kube-api-access-6987j\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.952498 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.952495 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ec09d40a-acdf-4b61-a322-a63fe33a75d4-ca\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.952720 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.952515 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ec09d40a-acdf-4b61-a322-a63fe33a75d4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.953187 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.953128 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec09d40a-acdf-4b61-a322-a63fe33a75d4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.955056 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.955030 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ec09d40a-acdf-4b61-a322-a63fe33a75d4-hub\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.955340 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.955298 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ec09d40a-acdf-4b61-a322-a63fe33a75d4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.955444 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.955402 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ec09d40a-acdf-4b61-a322-a63fe33a75d4-ca\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.955556 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.955538 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ec09d40a-acdf-4b61-a322-a63fe33a75d4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:09.959293 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.959270 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:09.970515 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:09.970469 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6987j\" (UniqueName: \"kubernetes.io/projected/ec09d40a-acdf-4b61-a322-a63fe33a75d4-kube-api-access-6987j\") pod \"cluster-proxy-proxy-agent-6db56cc59d-zs59v\" (UID: \"ec09d40a-acdf-4b61-a322-a63fe33a75d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:10.059089 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:10.059058 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" Jul 07 07:00:10.278595 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:10.278549 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:10.387891 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:10.387789 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:10.583141 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:10.583105 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:10.743265 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:10.743187 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:10.961820 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:10.961789 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs\") pod \"network-metrics-daemon-p79zw\" (UID: \"672040eb-b7da-43ca-be5d-1a8f98a9dab7\") " pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 07:00:10.961927 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:10.961894 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret\") pod \"global-pull-secret-syncer-6zrvs\" (UID: \"3fde7f49-f438-445e-bfc1-d645d3157b33\") " pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 07:00:10.964793 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:10.964770 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 07:00:10.965469 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:10.965452 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 07:00:10.974788 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:10.974769 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672040eb-b7da-43ca-be5d-1a8f98a9dab7-metrics-certs\") pod \"network-metrics-daemon-p79zw\" (UID: \"672040eb-b7da-43ca-be5d-1a8f98a9dab7\") " pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 07:00:10.974872 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:10.974838 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fde7f49-f438-445e-bfc1-d645d3157b33-original-pull-secret\") pod \"global-pull-secret-syncer-6zrvs\" (UID: \"3fde7f49-f438-445e-bfc1-d645d3157b33\") " pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 07:00:11.037447 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.037422 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v"] Jul 07 07:00:11.040754 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:00:11.040731 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec09d40a_acdf_4b61_a322_a63fe33a75d4.slice/crio-99c18467fcf9780b1972122a38df08c381605ff61e7926994246c5110c501ae3 WatchSource:0}: Error finding container 99c18467fcf9780b1972122a38df08c381605ff61e7926994246c5110c501ae3: Status 404 returned error can't find the container with id 99c18467fcf9780b1972122a38df08c381605ff61e7926994246c5110c501ae3 Jul 07 07:00:11.062806 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.062785 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdp2\" (UniqueName: \"kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2\") pod \"network-check-target-x74xm\" (UID: \"81061127-b732-4885-9f99-a9ad6c99301e\") " pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 07:00:11.065366 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.065347 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 07:00:11.076002 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.075979 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 07:00:11.086655 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.086630 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mdp2\" (UniqueName: \"kubernetes.io/projected/81061127-b732-4885-9f99-a9ad6c99301e-kube-api-access-9mdp2\") pod \"network-check-target-x74xm\" (UID: \"81061127-b732-4885-9f99-a9ad6c99301e\") " pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 07:00:11.112777 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.112745 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zrvs" Jul 07 07:00:11.216103 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.216081 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-59nzl\"" Jul 07 07:00:11.220233 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.220217 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2lkzt\"" Jul 07 07:00:11.224171 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.224157 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p79zw" Jul 07 07:00:11.229063 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.229048 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 07:00:11.249865 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.249437 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6zrvs"] Jul 07 07:00:11.252899 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:00:11.252877 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fde7f49_f438_445e_bfc1_d645d3157b33.slice/crio-3bc678e5ea5295a5880ce12a080c3eb8a74cd0f34ae8a63758f306cc24e35118 WatchSource:0}: Error finding container 3bc678e5ea5295a5880ce12a080c3eb8a74cd0f34ae8a63758f306cc24e35118: Status 404 returned error can't find the container with id 3bc678e5ea5295a5880ce12a080c3eb8a74cd0f34ae8a63758f306cc24e35118 Jul 07 07:00:11.374660 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.374628 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p79zw"] Jul 07 07:00:11.376987 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:00:11.376963 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod672040eb_b7da_43ca_be5d_1a8f98a9dab7.slice/crio-5c5b3414ec427fc7152478323fb66624bebcd0ec0fd8f1c9efafd0cebd5691f6 WatchSource:0}: Error finding container 5c5b3414ec427fc7152478323fb66624bebcd0ec0fd8f1c9efafd0cebd5691f6: Status 404 returned error can't find the container with id 5c5b3414ec427fc7152478323fb66624bebcd0ec0fd8f1c9efafd0cebd5691f6 Jul 07 07:00:11.380593 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.380580 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x74xm"] Jul 07 07:00:11.383134 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:00:11.383116 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81061127_b732_4885_9f99_a9ad6c99301e.slice/crio-abd8b483291bc365bbcf96209d7c815a3c09ea45be91986e46df4b5647b38330 WatchSource:0}: Error finding container abd8b483291bc365bbcf96209d7c815a3c09ea45be91986e46df4b5647b38330: Status 404 returned error can't find the container with id abd8b483291bc365bbcf96209d7c815a3c09ea45be91986e46df4b5647b38330 Jul 07 07:00:11.386555 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.386503 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:11.487758 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.487726 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:11.542018 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.541936 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x74xm" event={"ID":"81061127-b732-4885-9f99-a9ad6c99301e","Type":"ContainerStarted","Data":"abd8b483291bc365bbcf96209d7c815a3c09ea45be91986e46df4b5647b38330"} Jul 07 07:00:11.543981 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.543955 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9wst6" event={"ID":"9990b17f-cccd-4d39-a280-f179cf67e9c9","Type":"ContainerStarted","Data":"3b2c214007992f5170c2eaf0fbb7ac9a64ad63312d4ab407c4a65a261b852ca8"} Jul 07 07:00:11.545147 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.545127 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p79zw" event={"ID":"672040eb-b7da-43ca-be5d-1a8f98a9dab7","Type":"ContainerStarted","Data":"5c5b3414ec427fc7152478323fb66624bebcd0ec0fd8f1c9efafd0cebd5691f6"} Jul 07 07:00:11.546186 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.546166 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6zrvs" event={"ID":"3fde7f49-f438-445e-bfc1-d645d3157b33","Type":"ContainerStarted","Data":"3bc678e5ea5295a5880ce12a080c3eb8a74cd0f34ae8a63758f306cc24e35118"} Jul 07 07:00:11.547209 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.547191 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" event={"ID":"ec09d40a-acdf-4b61-a322-a63fe33a75d4","Type":"ContainerStarted","Data":"99c18467fcf9780b1972122a38df08c381605ff61e7926994246c5110c501ae3"} Jul 07 07:00:11.574978 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:11.574938 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9wst6" podStartSLOduration=17.278645975 podStartE2EDuration="24.574925062s" totalImagesPullingTime="7.296279087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:59:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:00:11.574209105 +0000 UTC m=+66.910175446" watchObservedRunningTime="2026-07-07 07:00:11.574925062 +0000 UTC m=+66.910891378" Jul 07 07:00:12.577048 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:12.577008 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-2xsrx\" (UID: \"27f2c73a-c551-4fe0-9563-00c9cd765437\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 07:00:12.583518 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:12.583483 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/27f2c73a-c551-4fe0-9563-00c9cd765437-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-2xsrx\" (UID: \"27f2c73a-c551-4fe0-9563-00c9cd765437\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 07:00:12.729509 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:12.729468 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6lh5g\"" Jul 07 07:00:12.741427 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:12.741395 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" Jul 07 07:00:12.985929 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:12.985843 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:13.453335 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:13.453276 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:15.805670 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:15.805637 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4g8xw\" (UID: \"19033e5d-d309-40af-a5dd-99322d4c8bd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" Jul 07 07:00:15.808451 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:15.808416 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/19033e5d-d309-40af-a5dd-99322d4c8bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4g8xw\" (UID: \"19033e5d-d309-40af-a5dd-99322d4c8bd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" Jul 07 07:00:15.816679 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:15.816655 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8kb8z\"" Jul 07 07:00:15.823202 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:15.823183 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" Jul 07 07:00:18.414562 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.414468 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx"] Jul 07 07:00:18.419646 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:00:18.419623 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27f2c73a_c551_4fe0_9563_00c9cd765437.slice/crio-1692ee752e68c914c1774654cd453df20e3f44bd20c9e8ff13d1b4613fd11fe5 WatchSource:0}: Error finding container 1692ee752e68c914c1774654cd453df20e3f44bd20c9e8ff13d1b4613fd11fe5: Status 404 returned error can't find the container with id 1692ee752e68c914c1774654cd453df20e3f44bd20c9e8ff13d1b4613fd11fe5 Jul 07 07:00:18.441977 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.439589 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw"] Jul 07 07:00:18.447208 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:00:18.447149 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19033e5d_d309_40af_a5dd_99322d4c8bd3.slice/crio-45fdf8a250dee3e0d81e7b88dc47c4988480697a9ae0e0c71847f5468506b736 WatchSource:0}: Error finding container 45fdf8a250dee3e0d81e7b88dc47c4988480697a9ae0e0c71847f5468506b736: Status 404 returned error can't find the container with id 45fdf8a250dee3e0d81e7b88dc47c4988480697a9ae0e0c71847f5468506b736 Jul 07 07:00:18.574997 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.574956 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p79zw" event={"ID":"672040eb-b7da-43ca-be5d-1a8f98a9dab7","Type":"ContainerStarted","Data":"3e2a56b9660213a0d4d02064385246c59ffc56e1daefecf0790e3de34f5e69f0"} Jul 07 07:00:18.575177 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.575017 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p79zw" event={"ID":"672040eb-b7da-43ca-be5d-1a8f98a9dab7","Type":"ContainerStarted","Data":"4a91f16bac878e581cd590d050cce9a1efcac7a5e04de0a17d4736877f693745"} Jul 07 07:00:18.576523 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.576495 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6zrvs" event={"ID":"3fde7f49-f438-445e-bfc1-d645d3157b33","Type":"ContainerStarted","Data":"7578f3df3c50775811aa43fe4ecf049ce270a3a2fc42bd461de80d7e6cd6b3e6"} Jul 07 07:00:18.577768 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.577742 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" event={"ID":"19033e5d-d309-40af-a5dd-99322d4c8bd3","Type":"ContainerStarted","Data":"45fdf8a250dee3e0d81e7b88dc47c4988480697a9ae0e0c71847f5468506b736"} Jul 07 07:00:18.578985 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.578962 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" event={"ID":"27f2c73a-c551-4fe0-9563-00c9cd765437","Type":"ContainerStarted","Data":"1692ee752e68c914c1774654cd453df20e3f44bd20c9e8ff13d1b4613fd11fe5"} Jul 07 07:00:18.583971 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.583951 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" event={"ID":"ec09d40a-acdf-4b61-a322-a63fe33a75d4","Type":"ContainerStarted","Data":"049d4534e5cf662ae0bfb45327402dda19f848ce6864dc361b66c5a9c82477bb"} Jul 07 07:00:18.584092 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.583982 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" event={"ID":"ec09d40a-acdf-4b61-a322-a63fe33a75d4","Type":"ContainerStarted","Data":"bafc8969bb230365dfeb10dd198d1fe46b5c64ed18ccc1d5bd3fdc86383d6158"} Jul 07 07:00:18.584092 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.583994 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" event={"ID":"ec09d40a-acdf-4b61-a322-a63fe33a75d4","Type":"ContainerStarted","Data":"523421ae04637f03fb448ab3a0421ccc9a4cb1f0de10ec8bf77952858d67acd7"} Jul 07 07:00:18.585503 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.585483 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x74xm" event={"ID":"81061127-b732-4885-9f99-a9ad6c99301e","Type":"ContainerStarted","Data":"3e85475cb811ad76f194902a3e357b96f1aad448641b1ce119996c6b205eb91b"} Jul 07 07:00:18.585630 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.585612 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 07:00:18.598509 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.598273 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-p79zw" podStartSLOduration=65.757051895 podStartE2EDuration="1m12.598257838s" totalImagesPullingTime="6.841205943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:00:18.598096261 +0000 UTC m=+73.934062612" watchObservedRunningTime="2026-07-07 07:00:18.598257838 +0000 UTC m=+73.934224155" Jul 07 07:00:18.620171 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.620131 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6zrvs" podStartSLOduration=65.638153593 podStartE2EDuration="1m12.620118648s" totalImagesPullingTime="6.981965055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:00:18.618907945 +0000 UTC m=+73.954874272" watchObservedRunningTime="2026-07-07 07:00:18.620118648 +0000 UTC m=+73.956084965" Jul 07 07:00:18.643822 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.643764 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6db56cc59d-zs59v" podStartSLOduration=2.447864096 podStartE2EDuration="9.643751903s" totalImagesPullingTime="7.195887807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:00:18.641931067 +0000 UTC m=+73.977897407" watchObservedRunningTime="2026-07-07 07:00:18.643751903 +0000 UTC m=+73.979718243" Jul 07 07:00:18.667829 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.667791 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-x74xm" podStartSLOduration=65.829648783 podStartE2EDuration="1m12.667779609s" totalImagesPullingTime="6.838130826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:00:18.666890146 +0000 UTC m=+74.002856464" watchObservedRunningTime="2026-07-07 07:00:18.667779609 +0000 UTC m=+74.003745963" Jul 07 07:00:18.760389 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.760365 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:18.864910 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.864882 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:18.970982 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:18.970954 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:20.594647 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:20.594611 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" event={"ID":"19033e5d-d309-40af-a5dd-99322d4c8bd3","Type":"ContainerStarted","Data":"f4e16c7bd5d1a26e68056afbacb98f1308e88c9159f3ffd69e451f1012c4fdb6"} Jul 07 07:00:20.596213 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:20.596177 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" event={"ID":"27f2c73a-c551-4fe0-9563-00c9cd765437","Type":"ContainerStarted","Data":"e847c7cd516065464f08535870ab87c7b6b82db49826c76a3c9b98282df4c753"} Jul 07 07:00:20.615198 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:20.615154 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-4g8xw" podStartSLOduration=35.8544691 podStartE2EDuration="37.615136718s" totalImagesPullingTime="1.760667618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:59:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:00:20.614584406 +0000 UTC m=+75.950550724" watchObservedRunningTime="2026-07-07 07:00:20.615136718 +0000 UTC m=+75.951103041" Jul 07 07:00:20.640685 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:20.640639 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-2xsrx" podStartSLOduration=38.848683615 podStartE2EDuration="40.640622819s" totalImagesPullingTime="1.791939204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:00:20.639679016 +0000 UTC m=+75.975645353" watchObservedRunningTime="2026-07-07 07:00:20.640622819 +0000 UTC m=+75.976589138" Jul 07 07:00:20.864741 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:20.864665 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:20.973197 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:20.973167 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:21.071951 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:21.071918 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:23.360410 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:23.360380 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:23.428976 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:23.428941 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:23.576642 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:23.576613 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:23.640272 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:23.640199 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:26.557169 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:26.557134 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:26.660631 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:26.660528 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:30.780473 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:30.780437 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:30.882532 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:30.882503 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:31.177226 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:31.177108 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:31.276494 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:31.276451 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:32.444746 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:32.444714 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:32.539592 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:32.539566 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:32.715393 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:32.715287 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:32.818247 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:32.818215 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:33.259182 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:33.259149 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:33.910452 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:33.910416 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:34.014692 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:34.014660 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:34.087003 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:34.086977 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:34.993377 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:34.993345 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:35.105691 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.105640 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:35.343009 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.342935 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:35.454484 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.454451 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vp4gn"] Jul 07 07:00:35.459388 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.459367 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.462087 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.462066 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 07:00:35.462818 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.462797 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 07:00:35.464407 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.464387 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bjwds\"" Jul 07 07:00:35.464546 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.464530 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 07:00:35.464753 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.464738 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 07:00:35.476446 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.476421 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:35.552784 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.552751 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/318f8939-2324-4c3b-9428-9e16f7487771-sys\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.552956 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.552811 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-527jg\" (UniqueName: \"kubernetes.io/projected/318f8939-2324-4c3b-9428-9e16f7487771-kube-api-access-527jg\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.552956 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.552843 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/318f8939-2324-4c3b-9428-9e16f7487771-root\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.552956 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.552865 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-tls\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.552956 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.552890 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/318f8939-2324-4c3b-9428-9e16f7487771-metrics-client-ca\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.553154 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.552956 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-wtmp\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.553154 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.552992 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-accelerators-collector-config\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.553154 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.553026 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.553154 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.553050 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-textfile\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.653431 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.653349 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-527jg\" (UniqueName: \"kubernetes.io/projected/318f8939-2324-4c3b-9428-9e16f7487771-kube-api-access-527jg\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.653431 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.653388 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/318f8939-2324-4c3b-9428-9e16f7487771-root\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.653431 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.653406 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-tls\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.653431 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.653430 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/318f8939-2324-4c3b-9428-9e16f7487771-metrics-client-ca\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.653763 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.653458 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-wtmp\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.653763 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.653481 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/318f8939-2324-4c3b-9428-9e16f7487771-root\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.653763 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.653495 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-accelerators-collector-config\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.653763 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:00:35.653544 2573 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 07:00:35.653763 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.653568 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.653763 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:00:35.653604 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-tls podName:318f8939-2324-4c3b-9428-9e16f7487771 nodeName:}" failed. No retries permitted until 2026-07-07 07:00:36.153582284 +0000 UTC m=+91.489548589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-tls") pod "node-exporter-vp4gn" (UID: "318f8939-2324-4c3b-9428-9e16f7487771") : secret "node-exporter-tls" not found Jul 07 07:00:35.653763 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.653634 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-textfile\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.653763 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.653686 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/318f8939-2324-4c3b-9428-9e16f7487771-sys\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.653763 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.653695 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-wtmp\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.653763 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.653768 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/318f8939-2324-4c3b-9428-9e16f7487771-sys\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.654244 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.653924 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-textfile\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.654244 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.654061 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-accelerators-collector-config\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.654244 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.654091 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/318f8939-2324-4c3b-9428-9e16f7487771-metrics-client-ca\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.656058 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.656040 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.666045 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.666019 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-527jg\" (UniqueName: \"kubernetes.io/projected/318f8939-2324-4c3b-9428-9e16f7487771-kube-api-access-527jg\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:35.775800 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.775767 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:35.876286 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:35.876256 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:36.139884 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:36.139847 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:36.157987 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:36.157961 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-tls\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:36.160211 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:36.160175 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/318f8939-2324-4c3b-9428-9e16f7487771-node-exporter-tls\") pod \"node-exporter-vp4gn\" (UID: \"318f8939-2324-4c3b-9428-9e16f7487771\") " pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:36.368764 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:36.368730 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vp4gn" Jul 07 07:00:36.381531 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:00:36.381490 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod318f8939_2324_4c3b_9428_9e16f7487771.slice/crio-885a5d391fab9ec1474c9dd0ab0c67bdc5632a7c17a4ddca31f94ab8a78b7cd3 WatchSource:0}: Error finding container 885a5d391fab9ec1474c9dd0ab0c67bdc5632a7c17a4ddca31f94ab8a78b7cd3: Status 404 returned error can't find the container with id 885a5d391fab9ec1474c9dd0ab0c67bdc5632a7c17a4ddca31f94ab8a78b7cd3 Jul 07 07:00:36.437567 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:36.437488 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:36.541448 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:36.541418 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:36.647097 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:36.647065 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:36.657988 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:36.657959 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vp4gn" event={"ID":"318f8939-2324-4c3b-9428-9e16f7487771","Type":"ContainerStarted","Data":"885a5d391fab9ec1474c9dd0ab0c67bdc5632a7c17a4ddca31f94ab8a78b7cd3"} Jul 07 07:00:36.856919 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:36.856893 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:36.950946 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:36.950890 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:37.662727 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:37.662687 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vp4gn" event={"ID":"318f8939-2324-4c3b-9428-9e16f7487771","Type":"ContainerStarted","Data":"2f2b294859c2467c76de3eee4a6e7445073d73633c36c1b636204e07ebc14b41"} Jul 07 07:00:38.327871 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:38.327837 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:38.429259 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:38.429226 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:38.585286 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:38.585210 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:38.667627 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:38.667597 2573 generic.go:356] "Generic (PLEG): container finished" podID="318f8939-2324-4c3b-9428-9e16f7487771" containerID="2f2b294859c2467c76de3eee4a6e7445073d73633c36c1b636204e07ebc14b41" exitCode=0 Jul 07 07:00:38.667983 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:38.667686 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vp4gn" event={"ID":"318f8939-2324-4c3b-9428-9e16f7487771","Type":"ContainerDied","Data":"2f2b294859c2467c76de3eee4a6e7445073d73633c36c1b636204e07ebc14b41"} Jul 07 07:00:38.686033 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:38.686010 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:39.672687 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:39.672651 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vp4gn" event={"ID":"318f8939-2324-4c3b-9428-9e16f7487771","Type":"ContainerStarted","Data":"be1ff3120d03a85c3171fe3344c851ceab124834e37bd0fdd5b8d5ad9d613bb4"} Jul 07 07:00:39.673089 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:39.672705 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vp4gn" event={"ID":"318f8939-2324-4c3b-9428-9e16f7487771","Type":"ContainerStarted","Data":"ce909041d9fc4319a3ca74314b05e75e4d58e266b72eaa1e0943fee992acef65"} Jul 07 07:00:39.696725 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:39.696673 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vp4gn" podStartSLOduration=3.4970887680000002 podStartE2EDuration="4.696658453s" totalImagesPullingTime="1.136061916s" totalInitContainerRuntime="63.507769ms" isStatefulPod=true podCreationTimestamp="2026-07-07 07:00:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:00:39.695622791 +0000 UTC m=+95.031589127" watchObservedRunningTime="2026-07-07 07:00:39.696658453 +0000 UTC m=+95.032624769" Jul 07 07:00:40.147263 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:40.147223 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:40.218965 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:40.218932 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:40.316899 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:40.316868 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:40.435205 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:40.435125 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:41.607410 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:41.607379 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:42.598573 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:42.598542 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:42.695008 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:42.694977 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:44.172513 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:44.172479 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:44.274518 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:44.274484 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:49.592179 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:49.592149 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-x74xm" Jul 07 07:00:50.940935 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:50.940906 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:51.037341 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:51.037298 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:51.121202 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:51.121166 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:51.857203 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:51.857172 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:51.918503 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:51.918473 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:53.751300 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:53.751266 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:53.828070 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:53.828043 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:57.600273 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:57.600238 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:57.676777 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:57.676750 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:59.944019 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:00:59.943983 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:00.040201 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:01:00.040157 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:06.765599 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:01:06.765562 2573 generic.go:356] "Generic (PLEG): container finished" podID="b490dc3b-4f35-4db0-be34-cc804b50c62f" containerID="9d2eba1008199829345d98169271bdee7cc0c7e9020661ec3040663ad530eb75" exitCode=0 Jul 07 07:01:06.765975 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:01:06.765617 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-j2872" event={"ID":"b490dc3b-4f35-4db0-be34-cc804b50c62f","Type":"ContainerDied","Data":"9d2eba1008199829345d98169271bdee7cc0c7e9020661ec3040663ad530eb75"} Jul 07 07:01:06.765975 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:01:06.765936 2573 scope.go:117] "RemoveContainer" containerID="9d2eba1008199829345d98169271bdee7cc0c7e9020661ec3040663ad530eb75" Jul 07 07:01:07.770846 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:01:07.770816 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-j2872" event={"ID":"b490dc3b-4f35-4db0-be34-cc804b50c62f","Type":"ContainerStarted","Data":"72c9b28e4409d4b673b68a308cb8a90d6df57c303ad1e8040a2e4dadb2ff8157"} Jul 07 07:01:12.633144 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:01:12.633108 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:12.712027 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:01:12.711999 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:32.158625 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:01:32.158589 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:32.258639 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:01:32.258611 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:45.998959 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:01:45.998922 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:54.645086 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:01:54.645046 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:55.236119 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:01:55.236088 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:03.833201 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:03.833118 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:03.905211 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:03.905182 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:08.518827 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:08.518786 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:09.888612 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:09.888578 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:09.949785 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:09.949756 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:11.043049 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:11.043014 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:11.128278 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:11.128248 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:14.339564 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:14.339527 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:14.410389 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:14.410363 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:14.630126 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:14.630052 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:14.695024 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:14.695002 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:21.896651 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:21.896611 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:21.996834 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:21.996798 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:25.355430 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:25.355396 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:25.456307 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:25.456274 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:25.524732 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:25.524705 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:27.612919 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:27.612889 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:27.705883 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:27.705853 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:37.558106 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:37.558071 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:37.618507 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:02:37.618469 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:03:36.772844 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:03:36.772765 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:03:36.868918 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:03:36.868888 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:04.277699 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:04.277661 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:05.015287 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:05.015256 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:04:05.015607 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:05.015589 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:04:05.025241 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:05.025221 2573 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 07:04:05.068265 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:05.068237 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:08.094679 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:08.094647 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:10.144837 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:10.144799 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:12.327943 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:12.327911 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:12.521295 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:12.521246 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:14.896246 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:14.896212 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:14.997011 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:14.996978 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:19.912578 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:19.912542 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:20.012293 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:20.012257 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:20.533278 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:20.533243 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:22.544565 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:22.544533 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-87nb4"] Jul 07 07:04:22.546432 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:22.546418 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-87nb4" Jul 07 07:04:22.549676 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:22.549659 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-9v5lv\"" Jul 07 07:04:22.551434 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:22.551413 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 07:04:22.551751 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:22.551735 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 07:04:22.564811 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:22.564790 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-87nb4"] Jul 07 07:04:22.599397 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:22.599373 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvxq6\" (UniqueName: \"kubernetes.io/projected/2fe61871-dda9-44d2-8076-8a9022838ea9-kube-api-access-rvxq6\") pod \"openshift-lws-operator-fd8ccff4c-87nb4\" (UID: \"2fe61871-dda9-44d2-8076-8a9022838ea9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-87nb4" Jul 07 07:04:22.599502 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:22.599408 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2fe61871-dda9-44d2-8076-8a9022838ea9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-87nb4\" (UID: \"2fe61871-dda9-44d2-8076-8a9022838ea9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-87nb4" Jul 07 07:04:22.700584 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:22.700558 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvxq6\" (UniqueName: \"kubernetes.io/projected/2fe61871-dda9-44d2-8076-8a9022838ea9-kube-api-access-rvxq6\") pod \"openshift-lws-operator-fd8ccff4c-87nb4\" (UID: \"2fe61871-dda9-44d2-8076-8a9022838ea9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-87nb4" Jul 07 07:04:22.700731 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:22.700594 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2fe61871-dda9-44d2-8076-8a9022838ea9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-87nb4\" (UID: \"2fe61871-dda9-44d2-8076-8a9022838ea9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-87nb4" Jul 07 07:04:22.701001 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:22.700981 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2fe61871-dda9-44d2-8076-8a9022838ea9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-87nb4\" (UID: \"2fe61871-dda9-44d2-8076-8a9022838ea9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-87nb4" Jul 07 07:04:22.709812 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:22.709791 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvxq6\" (UniqueName: \"kubernetes.io/projected/2fe61871-dda9-44d2-8076-8a9022838ea9-kube-api-access-rvxq6\") pod \"openshift-lws-operator-fd8ccff4c-87nb4\" (UID: \"2fe61871-dda9-44d2-8076-8a9022838ea9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-87nb4" Jul 07 07:04:22.854923 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:22.854850 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-87nb4" Jul 07 07:04:22.977897 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:22.977870 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-87nb4"] Jul 07 07:04:22.980926 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:04:22.980894 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fe61871_dda9_44d2_8076_8a9022838ea9.slice/crio-f1398906bb3b467c170c6b9fc01ac3911bf6e192a6b5c6ccf8cddcaf3c1515d0 WatchSource:0}: Error finding container f1398906bb3b467c170c6b9fc01ac3911bf6e192a6b5c6ccf8cddcaf3c1515d0: Status 404 returned error can't find the container with id f1398906bb3b467c170c6b9fc01ac3911bf6e192a6b5c6ccf8cddcaf3c1515d0 Jul 07 07:04:22.982376 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:22.982359 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 07:04:23.416459 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:23.416430 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-87nb4" event={"ID":"2fe61871-dda9-44d2-8076-8a9022838ea9","Type":"ContainerStarted","Data":"f1398906bb3b467c170c6b9fc01ac3911bf6e192a6b5c6ccf8cddcaf3c1515d0"} Jul 07 07:04:25.425739 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:25.425709 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-87nb4" event={"ID":"2fe61871-dda9-44d2-8076-8a9022838ea9","Type":"ContainerStarted","Data":"bfcca9f30a3ee9d6f70bc238195ef1000645160356183de66cb3f25189e5a181"} Jul 07 07:04:25.445462 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:25.445415 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-87nb4" podStartSLOduration=1.12539941 podStartE2EDuration="3.445399791s" totalImagesPullingTime="2.320000381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:04:25.443872096 +0000 UTC m=+320.779838413" watchObservedRunningTime="2026-07-07 07:04:25.445399791 +0000 UTC m=+320.781366107" Jul 07 07:04:26.220885 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:26.220838 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:26.319134 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:26.319099 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:28.182221 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:28.182191 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:39.747996 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:39.747937 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:40.497184 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:40.497145 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:40.635777 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:40.635739 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:41.122495 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.122448 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:41.397672 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.397582 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:41.523829 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.523798 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj"] Jul 07 07:04:41.523984 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.523797 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:41.527011 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.526996 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.529534 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.529505 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 07:04:41.529686 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.529668 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 07:04:41.529791 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.529775 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 07:04:41.529859 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.529781 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 07:04:41.529912 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.529861 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lpw58\"" Jul 07 07:04:41.536453 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.536433 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06c685c0-2e46-42be-b4c2-e4b65210432e-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbf9cc589-xr7bj\" (UID: \"06c685c0-2e46-42be-b4c2-e4b65210432e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.536575 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.536462 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnrr8\" (UniqueName: \"kubernetes.io/projected/06c685c0-2e46-42be-b4c2-e4b65210432e-kube-api-access-qnrr8\") pod \"opendatahub-operator-controller-manager-dbf9cc589-xr7bj\" (UID: \"06c685c0-2e46-42be-b4c2-e4b65210432e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.536575 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.536495 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06c685c0-2e46-42be-b4c2-e4b65210432e-tmp\") pod \"opendatahub-operator-controller-manager-dbf9cc589-xr7bj\" (UID: \"06c685c0-2e46-42be-b4c2-e4b65210432e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.536663 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.536613 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/06c685c0-2e46-42be-b4c2-e4b65210432e-manifests\") pod \"opendatahub-operator-controller-manager-dbf9cc589-xr7bj\" (UID: \"06c685c0-2e46-42be-b4c2-e4b65210432e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.536663 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.536642 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/06c685c0-2e46-42be-b4c2-e4b65210432e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbf9cc589-xr7bj\" (UID: \"06c685c0-2e46-42be-b4c2-e4b65210432e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.539990 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.539971 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj"] Jul 07 07:04:41.637442 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.637410 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/06c685c0-2e46-42be-b4c2-e4b65210432e-manifests\") pod \"opendatahub-operator-controller-manager-dbf9cc589-xr7bj\" (UID: \"06c685c0-2e46-42be-b4c2-e4b65210432e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.637442 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.637442 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/06c685c0-2e46-42be-b4c2-e4b65210432e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbf9cc589-xr7bj\" (UID: \"06c685c0-2e46-42be-b4c2-e4b65210432e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.637706 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.637470 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06c685c0-2e46-42be-b4c2-e4b65210432e-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbf9cc589-xr7bj\" (UID: \"06c685c0-2e46-42be-b4c2-e4b65210432e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.637706 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.637581 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnrr8\" (UniqueName: \"kubernetes.io/projected/06c685c0-2e46-42be-b4c2-e4b65210432e-kube-api-access-qnrr8\") pod \"opendatahub-operator-controller-manager-dbf9cc589-xr7bj\" (UID: \"06c685c0-2e46-42be-b4c2-e4b65210432e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.637706 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.637619 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06c685c0-2e46-42be-b4c2-e4b65210432e-tmp\") pod \"opendatahub-operator-controller-manager-dbf9cc589-xr7bj\" (UID: \"06c685c0-2e46-42be-b4c2-e4b65210432e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.637891 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.637872 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/06c685c0-2e46-42be-b4c2-e4b65210432e-manifests\") pod \"opendatahub-operator-controller-manager-dbf9cc589-xr7bj\" (UID: \"06c685c0-2e46-42be-b4c2-e4b65210432e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.639860 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.639832 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06c685c0-2e46-42be-b4c2-e4b65210432e-tmp\") pod \"opendatahub-operator-controller-manager-dbf9cc589-xr7bj\" (UID: \"06c685c0-2e46-42be-b4c2-e4b65210432e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.640039 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.640016 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06c685c0-2e46-42be-b4c2-e4b65210432e-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbf9cc589-xr7bj\" (UID: \"06c685c0-2e46-42be-b4c2-e4b65210432e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.640112 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.640098 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/06c685c0-2e46-42be-b4c2-e4b65210432e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbf9cc589-xr7bj\" (UID: \"06c685c0-2e46-42be-b4c2-e4b65210432e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.648911 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.648855 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnrr8\" (UniqueName: \"kubernetes.io/projected/06c685c0-2e46-42be-b4c2-e4b65210432e-kube-api-access-qnrr8\") pod \"opendatahub-operator-controller-manager-dbf9cc589-xr7bj\" (UID: \"06c685c0-2e46-42be-b4c2-e4b65210432e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.717459 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.717428 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:41.837873 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.837848 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:41.846017 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.845991 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:41.984919 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.984891 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj"] Jul 07 07:04:41.985955 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:04:41.985925 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06c685c0_2e46_42be_b4c2_e4b65210432e.slice/crio-7d691d80afb9243d7948d230cbbc772bf09fe556d7a522cec2e0b5933a55ea5e WatchSource:0}: Error finding container 7d691d80afb9243d7948d230cbbc772bf09fe556d7a522cec2e0b5933a55ea5e: Status 404 returned error can't find the container with id 7d691d80afb9243d7948d230cbbc772bf09fe556d7a522cec2e0b5933a55ea5e Jul 07 07:04:41.990396 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:41.990375 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:42.104451 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:42.104419 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:42.210648 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:42.210568 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:42.308819 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:42.308786 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:42.482422 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:42.482340 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" event={"ID":"06c685c0-2e46-42be-b4c2-e4b65210432e","Type":"ContainerStarted","Data":"7d691d80afb9243d7948d230cbbc772bf09fe556d7a522cec2e0b5933a55ea5e"} Jul 07 07:04:45.498044 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:45.498009 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" event={"ID":"06c685c0-2e46-42be-b4c2-e4b65210432e","Type":"ContainerStarted","Data":"6095882a270eea7bb95d252263e8e84629c10f189d226a6e58847099c3656c49"} Jul 07 07:04:45.598495 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:45.598465 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:45.715156 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:45.715131 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:46.503772 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:46.503739 2573 generic.go:356] "Generic (PLEG): container finished" podID="06c685c0-2e46-42be-b4c2-e4b65210432e" containerID="6095882a270eea7bb95d252263e8e84629c10f189d226a6e58847099c3656c49" exitCode=0 Jul 07 07:04:46.504135 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:46.503807 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" event={"ID":"06c685c0-2e46-42be-b4c2-e4b65210432e","Type":"ContainerDied","Data":"6095882a270eea7bb95d252263e8e84629c10f189d226a6e58847099c3656c49"} Jul 07 07:04:47.509973 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:47.509936 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" event={"ID":"06c685c0-2e46-42be-b4c2-e4b65210432e","Type":"ContainerStarted","Data":"ac915bcc3f9bbbcf54ae4b5686c2f393e9db154362461309d6715e4617c7254b"} Jul 07 07:04:47.510389 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:47.510047 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:04:47.537479 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:47.537420 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" podStartSLOduration=2.01970986 podStartE2EDuration="6.537398197s" totalImagesPullingTime="3.466856034s" totalInitContainerRuntime="1.050832303s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:04:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:04:47.535195867 +0000 UTC m=+342.871162185" watchObservedRunningTime="2026-07-07 07:04:47.537398197 +0000 UTC m=+342.873364517" Jul 07 07:04:49.085717 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:49.085680 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:49.152986 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:49.152953 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:58.516374 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:04:58.516276 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dbf9cc589-xr7bj" Jul 07 07:05:00.567336 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:00.567302 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:00.679432 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:00.679397 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:00.992122 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:00.992048 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:01.144212 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.144182 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt"] Jul 07 07:05:01.208822 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.208781 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt"] Jul 07 07:05:01.208999 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.208888 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt" Jul 07 07:05:01.211235 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.211214 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 07:05:01.211900 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.211880 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 07:05:01.212146 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.212126 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 07:05:01.212423 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.212407 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 07:05:01.212493 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.212419 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sbczf\"" Jul 07 07:05:01.284607 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.284574 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6bfr\" (UniqueName: \"kubernetes.io/projected/895e7b8b-350b-439b-abfd-351702707f22-kube-api-access-l6bfr\") pod \"kube-auth-proxy-54bd8c98d7-rj6wt\" (UID: \"895e7b8b-350b-439b-abfd-351702707f22\") " pod="openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt" Jul 07 07:05:01.284820 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.284677 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/895e7b8b-350b-439b-abfd-351702707f22-tls-certs\") pod \"kube-auth-proxy-54bd8c98d7-rj6wt\" (UID: \"895e7b8b-350b-439b-abfd-351702707f22\") " pod="openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt" Jul 07 07:05:01.284820 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.284709 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/895e7b8b-350b-439b-abfd-351702707f22-tmp\") pod \"kube-auth-proxy-54bd8c98d7-rj6wt\" (UID: \"895e7b8b-350b-439b-abfd-351702707f22\") " pod="openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt" Jul 07 07:05:01.310976 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.310942 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2j7jp"] Jul 07 07:05:01.315786 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.315766 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2j7jp" Jul 07 07:05:01.319482 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.319462 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 07 07:05:01.319924 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.319904 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-drh5b\"" Jul 07 07:05:01.326128 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.326100 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2j7jp"] Jul 07 07:05:01.378468 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.378435 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:01.385521 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.385491 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6bfr\" (UniqueName: \"kubernetes.io/projected/895e7b8b-350b-439b-abfd-351702707f22-kube-api-access-l6bfr\") pod \"kube-auth-proxy-54bd8c98d7-rj6wt\" (UID: \"895e7b8b-350b-439b-abfd-351702707f22\") " pod="openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt" Jul 07 07:05:01.385676 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.385539 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r48mb\" (UniqueName: \"kubernetes.io/projected/61c48a2b-f03d-4c37-9872-32773d868d00-kube-api-access-r48mb\") pod \"odh-model-controller-7998c67dc4-2j7jp\" (UID: \"61c48a2b-f03d-4c37-9872-32773d868d00\") " pod="opendatahub/odh-model-controller-7998c67dc4-2j7jp" Jul 07 07:05:01.385676 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.385594 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/895e7b8b-350b-439b-abfd-351702707f22-tls-certs\") pod \"kube-auth-proxy-54bd8c98d7-rj6wt\" (UID: \"895e7b8b-350b-439b-abfd-351702707f22\") " pod="openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt" Jul 07 07:05:01.385788 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:05:01.385720 2573 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 07 07:05:01.385837 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:05:01.385789 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/895e7b8b-350b-439b-abfd-351702707f22-tls-certs podName:895e7b8b-350b-439b-abfd-351702707f22 nodeName:}" failed. No retries permitted until 2026-07-07 07:05:01.885767064 +0000 UTC m=+357.221733364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/895e7b8b-350b-439b-abfd-351702707f22-tls-certs") pod "kube-auth-proxy-54bd8c98d7-rj6wt" (UID: "895e7b8b-350b-439b-abfd-351702707f22") : secret "kube-auth-proxy-tls" not found Jul 07 07:05:01.385837 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.385623 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61c48a2b-f03d-4c37-9872-32773d868d00-cert\") pod \"odh-model-controller-7998c67dc4-2j7jp\" (UID: \"61c48a2b-f03d-4c37-9872-32773d868d00\") " pod="opendatahub/odh-model-controller-7998c67dc4-2j7jp" Jul 07 07:05:01.385950 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.385848 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/895e7b8b-350b-439b-abfd-351702707f22-tmp\") pod \"kube-auth-proxy-54bd8c98d7-rj6wt\" (UID: \"895e7b8b-350b-439b-abfd-351702707f22\") " pod="openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt" Jul 07 07:05:01.388201 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.388181 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/895e7b8b-350b-439b-abfd-351702707f22-tmp\") pod \"kube-auth-proxy-54bd8c98d7-rj6wt\" (UID: \"895e7b8b-350b-439b-abfd-351702707f22\") " pod="openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt" Jul 07 07:05:01.394864 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.394833 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6bfr\" (UniqueName: \"kubernetes.io/projected/895e7b8b-350b-439b-abfd-351702707f22-kube-api-access-l6bfr\") pod \"kube-auth-proxy-54bd8c98d7-rj6wt\" (UID: \"895e7b8b-350b-439b-abfd-351702707f22\") " pod="openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt" Jul 07 07:05:01.486339 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.486282 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r48mb\" (UniqueName: \"kubernetes.io/projected/61c48a2b-f03d-4c37-9872-32773d868d00-kube-api-access-r48mb\") pod \"odh-model-controller-7998c67dc4-2j7jp\" (UID: \"61c48a2b-f03d-4c37-9872-32773d868d00\") " pod="opendatahub/odh-model-controller-7998c67dc4-2j7jp" Jul 07 07:05:01.486529 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.486382 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61c48a2b-f03d-4c37-9872-32773d868d00-cert\") pod \"odh-model-controller-7998c67dc4-2j7jp\" (UID: \"61c48a2b-f03d-4c37-9872-32773d868d00\") " pod="opendatahub/odh-model-controller-7998c67dc4-2j7jp" Jul 07 07:05:01.486529 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.486431 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:01.486529 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:05:01.486497 2573 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 07 07:05:01.486699 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:05:01.486571 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61c48a2b-f03d-4c37-9872-32773d868d00-cert podName:61c48a2b-f03d-4c37-9872-32773d868d00 nodeName:}" failed. No retries permitted until 2026-07-07 07:05:01.986552854 +0000 UTC m=+357.322519150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/61c48a2b-f03d-4c37-9872-32773d868d00-cert") pod "odh-model-controller-7998c67dc4-2j7jp" (UID: "61c48a2b-f03d-4c37-9872-32773d868d00") : secret "odh-model-controller-webhook-cert" not found Jul 07 07:05:01.500658 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.500632 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r48mb\" (UniqueName: \"kubernetes.io/projected/61c48a2b-f03d-4c37-9872-32773d868d00-kube-api-access-r48mb\") pod \"odh-model-controller-7998c67dc4-2j7jp\" (UID: \"61c48a2b-f03d-4c37-9872-32773d868d00\") " pod="opendatahub/odh-model-controller-7998c67dc4-2j7jp" Jul 07 07:05:01.591576 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.591482 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:01.755291 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.755256 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:01.867644 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.867518 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:01.890767 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.890723 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/895e7b8b-350b-439b-abfd-351702707f22-tls-certs\") pod \"kube-auth-proxy-54bd8c98d7-rj6wt\" (UID: \"895e7b8b-350b-439b-abfd-351702707f22\") " pod="openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt" Jul 07 07:05:01.893363 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.893312 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/895e7b8b-350b-439b-abfd-351702707f22-tls-certs\") pod \"kube-auth-proxy-54bd8c98d7-rj6wt\" (UID: \"895e7b8b-350b-439b-abfd-351702707f22\") " pod="openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt" Jul 07 07:05:01.991469 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.991436 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61c48a2b-f03d-4c37-9872-32773d868d00-cert\") pod \"odh-model-controller-7998c67dc4-2j7jp\" (UID: \"61c48a2b-f03d-4c37-9872-32773d868d00\") " pod="opendatahub/odh-model-controller-7998c67dc4-2j7jp" Jul 07 07:05:01.993825 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:01.993805 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61c48a2b-f03d-4c37-9872-32773d868d00-cert\") pod \"odh-model-controller-7998c67dc4-2j7jp\" (UID: \"61c48a2b-f03d-4c37-9872-32773d868d00\") " pod="opendatahub/odh-model-controller-7998c67dc4-2j7jp" Jul 07 07:05:02.019563 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.019531 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:02.119107 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.119023 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt" Jul 07 07:05:02.127190 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.127167 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:02.228923 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.228863 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2j7jp" Jul 07 07:05:02.270155 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.270075 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt"] Jul 07 07:05:02.273523 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:05:02.273494 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod895e7b8b_350b_439b_abfd_351702707f22.slice/crio-65ecc5b10464c8359217a18c18a955a1656440a459b387908390c227da49cde0 WatchSource:0}: Error finding container 65ecc5b10464c8359217a18c18a955a1656440a459b387908390c227da49cde0: Status 404 returned error can't find the container with id 65ecc5b10464c8359217a18c18a955a1656440a459b387908390c227da49cde0 Jul 07 07:05:02.378677 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.378588 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2j7jp"] Jul 07 07:05:02.382241 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:05:02.382215 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61c48a2b_f03d_4c37_9872_32773d868d00.slice/crio-6d3672c1398734a6b45144c6370f5b3fc4d199324054e53944e6f29586775087 WatchSource:0}: Error finding container 6d3672c1398734a6b45144c6370f5b3fc4d199324054e53944e6f29586775087: Status 404 returned error can't find the container with id 6d3672c1398734a6b45144c6370f5b3fc4d199324054e53944e6f29586775087 Jul 07 07:05:02.419091 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.419058 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gdsdv"] Jul 07 07:05:02.421751 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.421734 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gdsdv" Jul 07 07:05:02.424425 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.424404 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 07 07:05:02.435125 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.435094 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gdsdv"] Jul 07 07:05:02.496176 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.496140 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b123dbe-7add-499e-a759-0bd4537e13e8-tls-certs\") pod \"model-serving-api-54d89bd5fc-gdsdv\" (UID: \"9b123dbe-7add-499e-a759-0bd4537e13e8\") " pod="opendatahub/model-serving-api-54d89bd5fc-gdsdv" Jul 07 07:05:02.496391 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.496183 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b4b4\" (UniqueName: \"kubernetes.io/projected/9b123dbe-7add-499e-a759-0bd4537e13e8-kube-api-access-8b4b4\") pod \"model-serving-api-54d89bd5fc-gdsdv\" (UID: \"9b123dbe-7add-499e-a759-0bd4537e13e8\") " pod="opendatahub/model-serving-api-54d89bd5fc-gdsdv" Jul 07 07:05:02.513065 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.513036 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7czgj"] Jul 07 07:05:02.515181 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.515166 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.518182 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.518159 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 07:05:02.518182 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.518180 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 07:05:02.518451 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.518434 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 07:05:02.518537 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.518455 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 07:05:02.518537 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.518504 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-n2n8g\"" Jul 07 07:05:02.530177 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.530125 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:02.531462 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.531441 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7czgj"] Jul 07 07:05:02.568950 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.568907 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2j7jp" event={"ID":"61c48a2b-f03d-4c37-9872-32773d868d00","Type":"ContainerStarted","Data":"6d3672c1398734a6b45144c6370f5b3fc4d199324054e53944e6f29586775087"} Jul 07 07:05:02.570225 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.570197 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt" event={"ID":"895e7b8b-350b-439b-abfd-351702707f22","Type":"ContainerStarted","Data":"65ecc5b10464c8359217a18c18a955a1656440a459b387908390c227da49cde0"} Jul 07 07:05:02.597648 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.597617 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b123dbe-7add-499e-a759-0bd4537e13e8-tls-certs\") pod \"model-serving-api-54d89bd5fc-gdsdv\" (UID: \"9b123dbe-7add-499e-a759-0bd4537e13e8\") " pod="opendatahub/model-serving-api-54d89bd5fc-gdsdv" Jul 07 07:05:02.598018 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.597658 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-cacerts\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.598018 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.597690 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b4b4\" (UniqueName: \"kubernetes.io/projected/9b123dbe-7add-499e-a759-0bd4537e13e8-kube-api-access-8b4b4\") pod \"model-serving-api-54d89bd5fc-gdsdv\" (UID: \"9b123dbe-7add-499e-a759-0bd4537e13e8\") " pod="opendatahub/model-serving-api-54d89bd5fc-gdsdv" Jul 07 07:05:02.598018 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:05:02.597767 2573 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 07 07:05:02.598018 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.597785 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.598018 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:05:02.597823 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b123dbe-7add-499e-a759-0bd4537e13e8-tls-certs podName:9b123dbe-7add-499e-a759-0bd4537e13e8 nodeName:}" failed. No retries permitted until 2026-07-07 07:05:03.097806605 +0000 UTC m=+358.433772907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9b123dbe-7add-499e-a759-0bd4537e13e8-tls-certs") pod "model-serving-api-54d89bd5fc-gdsdv" (UID: "9b123dbe-7add-499e-a759-0bd4537e13e8") : secret "model-serving-api-tls" not found Jul 07 07:05:02.598018 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.597869 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-token\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.598018 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.597893 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7gdn\" (UniqueName: \"kubernetes.io/projected/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-kube-api-access-t7gdn\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.598018 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.597919 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-local-certs\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.598018 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.597954 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.598018 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.598009 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.609588 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.609565 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b4b4\" (UniqueName: \"kubernetes.io/projected/9b123dbe-7add-499e-a759-0bd4537e13e8-kube-api-access-8b4b4\") pod \"model-serving-api-54d89bd5fc-gdsdv\" (UID: \"9b123dbe-7add-499e-a759-0bd4537e13e8\") " pod="opendatahub/model-serving-api-54d89bd5fc-gdsdv" Jul 07 07:05:02.641869 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.641776 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:02.699217 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.699185 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.699217 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.699230 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-token\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.699513 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.699254 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7gdn\" (UniqueName: \"kubernetes.io/projected/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-kube-api-access-t7gdn\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.699513 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.699282 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-local-certs\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.699513 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.699442 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.699513 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.699498 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.699720 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.699561 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-cacerts\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.699950 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.699925 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.701888 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.701857 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.701986 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.701904 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.702029 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.702005 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-local-certs\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.702360 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.702315 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-cacerts\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.716441 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.716416 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-token\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.716713 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.716691 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7gdn\" (UniqueName: \"kubernetes.io/projected/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-kube-api-access-t7gdn\") pod \"istiod-openshift-gateway-7574459656-7czgj\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.824817 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.824779 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:02.867027 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.866986 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:02.973761 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:02.973726 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7czgj"] Jul 07 07:05:02.976643 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:05:02.976610 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25e409ee_a5df_4d7a_b2ce_815c1ac600d6.slice/crio-9e36318de894ad7d14ce42ceef0478b15bdfd87fd511d117b98b6d749c411067 WatchSource:0}: Error finding container 9e36318de894ad7d14ce42ceef0478b15bdfd87fd511d117b98b6d749c411067: Status 404 returned error can't find the container with id 9e36318de894ad7d14ce42ceef0478b15bdfd87fd511d117b98b6d749c411067 Jul 07 07:05:03.102672 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:03.102627 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b123dbe-7add-499e-a759-0bd4537e13e8-tls-certs\") pod \"model-serving-api-54d89bd5fc-gdsdv\" (UID: \"9b123dbe-7add-499e-a759-0bd4537e13e8\") " pod="opendatahub/model-serving-api-54d89bd5fc-gdsdv" Jul 07 07:05:03.105448 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:03.105423 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b123dbe-7add-499e-a759-0bd4537e13e8-tls-certs\") pod \"model-serving-api-54d89bd5fc-gdsdv\" (UID: \"9b123dbe-7add-499e-a759-0bd4537e13e8\") " pod="opendatahub/model-serving-api-54d89bd5fc-gdsdv" Jul 07 07:05:03.332510 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:03.332479 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gdsdv" Jul 07 07:05:03.524828 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:03.524748 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gdsdv"] Jul 07 07:05:03.581504 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:03.581419 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gdsdv" event={"ID":"9b123dbe-7add-499e-a759-0bd4537e13e8","Type":"ContainerStarted","Data":"2aa840450d9254b813971a4a31632a2990a9b4e3d8e9abde4f10230dc62b346f"} Jul 07 07:05:03.583974 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:03.583857 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" event={"ID":"25e409ee-a5df-4d7a-b2ce-815c1ac600d6","Type":"ContainerStarted","Data":"9e36318de894ad7d14ce42ceef0478b15bdfd87fd511d117b98b6d749c411067"} Jul 07 07:05:03.763102 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:03.763009 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:07.926124 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:07.926099 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 07:05:07.926617 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:07.926559 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 07:05:07.926693 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:07.926674 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 07:05:08.614823 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:08.614782 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt" event={"ID":"895e7b8b-350b-439b-abfd-351702707f22","Type":"ContainerStarted","Data":"93181f9bbf78ca142d2ab40e804c31cba53fcdc02c5ffeca803bae2852d0cedb"} Jul 07 07:05:08.616406 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:08.616375 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2j7jp" event={"ID":"61c48a2b-f03d-4c37-9872-32773d868d00","Type":"ContainerStarted","Data":"3d8936d0f88da3f0fcabbcfdf7ba8510621d4a90b71d3840a6cf4c1ac06b7c19"} Jul 07 07:05:08.616556 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:08.616454 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2j7jp" Jul 07 07:05:08.617823 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:08.617802 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gdsdv" event={"ID":"9b123dbe-7add-499e-a759-0bd4537e13e8","Type":"ContainerStarted","Data":"a7d5a1c63f81a2b1b601620f887531c8acbab4c41fd6fd2c4f215776d0a71068"} Jul 07 07:05:08.617931 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:08.617851 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-gdsdv" Jul 07 07:05:08.619291 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:08.619274 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" event={"ID":"25e409ee-a5df-4d7a-b2ce-815c1ac600d6","Type":"ContainerStarted","Data":"a1342d9b29a6e765dbe37c3fd85e21f81825f916d912684598cedfd16a4043ad"} Jul 07 07:05:08.619463 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:08.619451 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:08.658781 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:08.658738 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-54bd8c98d7-rj6wt" podStartSLOduration=2.012447115 podStartE2EDuration="7.658726703s" totalImagesPullingTime="5.646279588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:05:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:05:08.65813871 +0000 UTC m=+363.994105047" watchObservedRunningTime="2026-07-07 07:05:08.658726703 +0000 UTC m=+363.994693021" Jul 07 07:05:08.709080 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:08.709037 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-gdsdv" podStartSLOduration=2.315944816 podStartE2EDuration="6.709017637s" totalImagesPullingTime="4.393072821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:05:08.707585042 +0000 UTC m=+364.043551383" watchObservedRunningTime="2026-07-07 07:05:08.709017637 +0000 UTC m=+364.044983953" Jul 07 07:05:08.756648 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:08.756579 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" podStartSLOduration=1.8126944790000001 podStartE2EDuration="6.756557425s" totalImagesPullingTime="4.943862946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:05:08.754543878 +0000 UTC m=+364.090510238" watchObservedRunningTime="2026-07-07 07:05:08.756557425 +0000 UTC m=+364.092523744" Jul 07 07:05:08.864068 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:08.864031 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:09.072905 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:09.072873 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:09.624730 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:09.624695 2573 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-7czgj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 07:05:09.624883 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:09.624750 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" podUID="25e409ee-a5df-4d7a-b2ce-815c1ac600d6" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 07:05:09.838912 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:09.838879 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:10.083637 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.083607 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:10.206522 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.206485 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:10.348131 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.348024 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-2j7jp" podStartSLOduration=3.75579607 podStartE2EDuration="9.348006111s" totalImagesPullingTime="5.592210041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:05:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:05:08.79803354 +0000 UTC m=+364.133999877" watchObservedRunningTime="2026-07-07 07:05:10.348006111 +0000 UTC m=+365.683972428" Jul 07 07:05:10.348602 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.348579 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh"] Jul 07 07:05:10.351871 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.351850 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.367908 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.367864 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh"] Jul 07 07:05:10.476933 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.476902 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c8987396-97f4-4963-86e5-3634adc1aec4-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.477097 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.476949 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8987396-97f4-4963-86e5-3634adc1aec4-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.477097 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.476967 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8987396-97f4-4963-86e5-3634adc1aec4-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.477097 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.476988 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c8987396-97f4-4963-86e5-3634adc1aec4-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.477097 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.477048 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c8987396-97f4-4963-86e5-3634adc1aec4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.477097 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.477090 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c8987396-97f4-4963-86e5-3634adc1aec4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.477255 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.477129 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtt5p\" (UniqueName: \"kubernetes.io/projected/c8987396-97f4-4963-86e5-3634adc1aec4-kube-api-access-vtt5p\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.577596 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.577569 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8987396-97f4-4963-86e5-3634adc1aec4-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.577596 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.577600 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8987396-97f4-4963-86e5-3634adc1aec4-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.577789 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.577616 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c8987396-97f4-4963-86e5-3634adc1aec4-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.577789 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.577638 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c8987396-97f4-4963-86e5-3634adc1aec4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.577789 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.577698 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c8987396-97f4-4963-86e5-3634adc1aec4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.577789 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.577751 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtt5p\" (UniqueName: \"kubernetes.io/projected/c8987396-97f4-4963-86e5-3634adc1aec4-kube-api-access-vtt5p\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.577982 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.577797 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c8987396-97f4-4963-86e5-3634adc1aec4-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.578455 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.578428 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c8987396-97f4-4963-86e5-3634adc1aec4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.580031 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.580001 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8987396-97f4-4963-86e5-3634adc1aec4-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.580031 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.580024 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c8987396-97f4-4963-86e5-3634adc1aec4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.580209 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.580116 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c8987396-97f4-4963-86e5-3634adc1aec4-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.580209 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.580147 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c8987396-97f4-4963-86e5-3634adc1aec4-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.587052 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.587028 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8987396-97f4-4963-86e5-3634adc1aec4-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.587839 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.587823 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtt5p\" (UniqueName: \"kubernetes.io/projected/c8987396-97f4-4963-86e5-3634adc1aec4-kube-api-access-vtt5p\") pod \"istiod-openshift-gateway-94bb8fbfd-l6hdh\" (UID: \"c8987396-97f4-4963-86e5-3634adc1aec4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.661473 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.661407 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:10.804069 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.804046 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh"] Jul 07 07:05:10.804584 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:05:10.804550 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8987396_97f4_4963_86e5_3634adc1aec4.slice/crio-06fdb03862e47715f27d4eac49b74e5f6d10462c508414b8dbd1b72fffc4dcb3 WatchSource:0}: Error finding container 06fdb03862e47715f27d4eac49b74e5f6d10462c508414b8dbd1b72fffc4dcb3: Status 404 returned error can't find the container with id 06fdb03862e47715f27d4eac49b74e5f6d10462c508414b8dbd1b72fffc4dcb3 Jul 07 07:05:10.807871 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.807837 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 07:05:10.807987 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:10.807900 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 07:05:11.631961 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:11.631924 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" event={"ID":"c8987396-97f4-4963-86e5-3634adc1aec4","Type":"ContainerStarted","Data":"233b72f4ccbcf589fba572349b8f841fc1980cfdbe0fead1ffaae25b313d139c"} Jul 07 07:05:11.632444 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:11.631977 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" event={"ID":"c8987396-97f4-4963-86e5-3634adc1aec4","Type":"ContainerStarted","Data":"06fdb03862e47715f27d4eac49b74e5f6d10462c508414b8dbd1b72fffc4dcb3"} Jul 07 07:05:11.632444 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:11.632089 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:11.633783 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:11.633758 2573 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-l6hdh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 07:05:11.633886 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:11.633805 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" podUID="c8987396-97f4-4963-86e5-3634adc1aec4" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 07:05:11.668772 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:11.668725 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" podStartSLOduration=1.6687106310000002 podStartE2EDuration="1.668710631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:05:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:05:11.668158952 +0000 UTC m=+367.004125269" watchObservedRunningTime="2026-07-07 07:05:11.668710631 +0000 UTC m=+367.004676947" Jul 07 07:05:12.623795 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:12.623762 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:12.638736 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:12.638706 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l6hdh" Jul 07 07:05:12.722660 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:12.722627 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7czgj"] Jul 07 07:05:12.723354 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:12.723124 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" podUID="25e409ee-a5df-4d7a-b2ce-815c1ac600d6" containerName="discovery" containerID="cri-o://a1342d9b29a6e765dbe37c3fd85e21f81825f916d912684598cedfd16a4043ad" gracePeriod=30 Jul 07 07:05:12.962960 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:12.962937 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:13.099431 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.099401 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-token\") pod \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " Jul 07 07:05:13.099431 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.099433 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-kubeconfig\") pod \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " Jul 07 07:05:13.099643 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.099485 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-csr-dns-cert\") pod \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " Jul 07 07:05:13.099643 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.099521 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-csr-ca-configmap\") pod \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " Jul 07 07:05:13.099643 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.099560 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7gdn\" (UniqueName: \"kubernetes.io/projected/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-kube-api-access-t7gdn\") pod \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " Jul 07 07:05:13.099643 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.099587 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-cacerts\") pod \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " Jul 07 07:05:13.099643 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.099621 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-local-certs\") pod \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\" (UID: \"25e409ee-a5df-4d7a-b2ce-815c1ac600d6\") " Jul 07 07:05:13.099955 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.099922 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "25e409ee-a5df-4d7a-b2ce-815c1ac600d6" (UID: "25e409ee-a5df-4d7a-b2ce-815c1ac600d6"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:05:13.101835 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.101774 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "25e409ee-a5df-4d7a-b2ce-815c1ac600d6" (UID: "25e409ee-a5df-4d7a-b2ce-815c1ac600d6"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:05:13.101929 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.101872 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-token" (OuterVolumeSpecName: "istio-token") pod "25e409ee-a5df-4d7a-b2ce-815c1ac600d6" (UID: "25e409ee-a5df-4d7a-b2ce-815c1ac600d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:05:13.102090 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.102061 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-kube-api-access-t7gdn" (OuterVolumeSpecName: "kube-api-access-t7gdn") pod "25e409ee-a5df-4d7a-b2ce-815c1ac600d6" (UID: "25e409ee-a5df-4d7a-b2ce-815c1ac600d6"). InnerVolumeSpecName "kube-api-access-t7gdn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:05:13.102167 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.102102 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-cacerts" (OuterVolumeSpecName: "cacerts") pod "25e409ee-a5df-4d7a-b2ce-815c1ac600d6" (UID: "25e409ee-a5df-4d7a-b2ce-815c1ac600d6"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:05:13.102220 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.102175 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "25e409ee-a5df-4d7a-b2ce-815c1ac600d6" (UID: "25e409ee-a5df-4d7a-b2ce-815c1ac600d6"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:05:13.102258 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.102236 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-local-certs" (OuterVolumeSpecName: "local-certs") pod "25e409ee-a5df-4d7a-b2ce-815c1ac600d6" (UID: "25e409ee-a5df-4d7a-b2ce-815c1ac600d6"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:05:13.201288 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.201203 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-token\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:05:13.201288 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.201231 2573 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-kubeconfig\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:05:13.201288 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.201243 2573 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-csr-dns-cert\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:05:13.201288 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.201257 2573 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-istio-csr-ca-configmap\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:05:13.201288 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.201269 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7gdn\" (UniqueName: \"kubernetes.io/projected/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-kube-api-access-t7gdn\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:05:13.201288 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.201280 2573 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-cacerts\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:05:13.201288 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.201292 2573 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/25e409ee-a5df-4d7a-b2ce-815c1ac600d6-local-certs\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:05:13.521184 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.521152 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:13.563737 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.563707 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-twmpt"] Jul 07 07:05:13.564024 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.564011 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25e409ee-a5df-4d7a-b2ce-815c1ac600d6" containerName="discovery" Jul 07 07:05:13.564068 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.564027 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="25e409ee-a5df-4d7a-b2ce-815c1ac600d6" containerName="discovery" Jul 07 07:05:13.564102 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.564073 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="25e409ee-a5df-4d7a-b2ce-815c1ac600d6" containerName="discovery" Jul 07 07:05:13.566716 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.566700 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-twmpt" Jul 07 07:05:13.569813 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.569646 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 07 07:05:13.569813 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.569676 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-x9nxn\"" Jul 07 07:05:13.580795 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.580775 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p"] Jul 07 07:05:13.583794 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.583774 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p" Jul 07 07:05:13.586667 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.586402 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 07 07:05:13.586797 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.586772 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-twmpt"] Jul 07 07:05:13.586902 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.586837 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-2dhnd\"" Jul 07 07:05:13.613429 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.613363 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p"] Jul 07 07:05:13.629970 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.629939 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:13.644389 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.644346 2573 generic.go:356] "Generic (PLEG): container finished" podID="25e409ee-a5df-4d7a-b2ce-815c1ac600d6" containerID="a1342d9b29a6e765dbe37c3fd85e21f81825f916d912684598cedfd16a4043ad" exitCode=0 Jul 07 07:05:13.644876 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.644410 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" Jul 07 07:05:13.644876 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.644474 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" event={"ID":"25e409ee-a5df-4d7a-b2ce-815c1ac600d6","Type":"ContainerDied","Data":"a1342d9b29a6e765dbe37c3fd85e21f81825f916d912684598cedfd16a4043ad"} Jul 07 07:05:13.644876 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.644538 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7czgj" event={"ID":"25e409ee-a5df-4d7a-b2ce-815c1ac600d6","Type":"ContainerDied","Data":"9e36318de894ad7d14ce42ceef0478b15bdfd87fd511d117b98b6d749c411067"} Jul 07 07:05:13.644876 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.644565 2573 scope.go:117] "RemoveContainer" containerID="a1342d9b29a6e765dbe37c3fd85e21f81825f916d912684598cedfd16a4043ad" Jul 07 07:05:13.654107 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.654087 2573 scope.go:117] "RemoveContainer" containerID="a1342d9b29a6e765dbe37c3fd85e21f81825f916d912684598cedfd16a4043ad" Jul 07 07:05:13.654431 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:05:13.654409 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1342d9b29a6e765dbe37c3fd85e21f81825f916d912684598cedfd16a4043ad\": container with ID starting with a1342d9b29a6e765dbe37c3fd85e21f81825f916d912684598cedfd16a4043ad not found: ID does not exist" containerID="a1342d9b29a6e765dbe37c3fd85e21f81825f916d912684598cedfd16a4043ad" Jul 07 07:05:13.654516 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.654439 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1342d9b29a6e765dbe37c3fd85e21f81825f916d912684598cedfd16a4043ad"} err="failed to get container status \"a1342d9b29a6e765dbe37c3fd85e21f81825f916d912684598cedfd16a4043ad\": rpc error: code = NotFound desc = could not find container \"a1342d9b29a6e765dbe37c3fd85e21f81825f916d912684598cedfd16a4043ad\": container with ID starting with a1342d9b29a6e765dbe37c3fd85e21f81825f916d912684598cedfd16a4043ad not found: ID does not exist" Jul 07 07:05:13.675391 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.675348 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7czgj"] Jul 07 07:05:13.677111 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.677090 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7czgj"] Jul 07 07:05:13.714546 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.714525 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1b1f08e-b37d-4cec-a942-87795b69dfec-cert\") pod \"kserve-controller-manager-d6f4b65b8-twmpt\" (UID: \"e1b1f08e-b37d-4cec-a942-87795b69dfec\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-twmpt" Jul 07 07:05:13.714656 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.714555 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vws5l\" (UniqueName: \"kubernetes.io/projected/e1b1f08e-b37d-4cec-a942-87795b69dfec-kube-api-access-vws5l\") pod \"kserve-controller-manager-d6f4b65b8-twmpt\" (UID: \"e1b1f08e-b37d-4cec-a942-87795b69dfec\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-twmpt" Jul 07 07:05:13.714656 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.714606 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b0f2bfd-706a-46ec-a92e-a02c5afe9b15-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gdc4p\" (UID: \"0b0f2bfd-706a-46ec-a92e-a02c5afe9b15\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p" Jul 07 07:05:13.714766 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.714682 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6572\" (UniqueName: \"kubernetes.io/projected/0b0f2bfd-706a-46ec-a92e-a02c5afe9b15-kube-api-access-f6572\") pod \"llmisvc-controller-manager-777c6bd7d5-gdc4p\" (UID: \"0b0f2bfd-706a-46ec-a92e-a02c5afe9b15\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p" Jul 07 07:05:13.815681 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.815602 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b0f2bfd-706a-46ec-a92e-a02c5afe9b15-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gdc4p\" (UID: \"0b0f2bfd-706a-46ec-a92e-a02c5afe9b15\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p" Jul 07 07:05:13.815681 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.815650 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6572\" (UniqueName: \"kubernetes.io/projected/0b0f2bfd-706a-46ec-a92e-a02c5afe9b15-kube-api-access-f6572\") pod \"llmisvc-controller-manager-777c6bd7d5-gdc4p\" (UID: \"0b0f2bfd-706a-46ec-a92e-a02c5afe9b15\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p" Jul 07 07:05:13.815894 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.815766 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1b1f08e-b37d-4cec-a942-87795b69dfec-cert\") pod \"kserve-controller-manager-d6f4b65b8-twmpt\" (UID: \"e1b1f08e-b37d-4cec-a942-87795b69dfec\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-twmpt" Jul 07 07:05:13.815894 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.815802 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vws5l\" (UniqueName: \"kubernetes.io/projected/e1b1f08e-b37d-4cec-a942-87795b69dfec-kube-api-access-vws5l\") pod \"kserve-controller-manager-d6f4b65b8-twmpt\" (UID: \"e1b1f08e-b37d-4cec-a942-87795b69dfec\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-twmpt" Jul 07 07:05:13.816035 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:05:13.816015 2573 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 07 07:05:13.816100 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:05:13.816086 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b0f2bfd-706a-46ec-a92e-a02c5afe9b15-cert podName:0b0f2bfd-706a-46ec-a92e-a02c5afe9b15 nodeName:}" failed. No retries permitted until 2026-07-07 07:05:14.316065497 +0000 UTC m=+369.652031800 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0b0f2bfd-706a-46ec-a92e-a02c5afe9b15-cert") pod "llmisvc-controller-manager-777c6bd7d5-gdc4p" (UID: "0b0f2bfd-706a-46ec-a92e-a02c5afe9b15") : secret "llmisvc-webhook-server-cert" not found Jul 07 07:05:13.818147 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.818130 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1b1f08e-b37d-4cec-a942-87795b69dfec-cert\") pod \"kserve-controller-manager-d6f4b65b8-twmpt\" (UID: \"e1b1f08e-b37d-4cec-a942-87795b69dfec\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-twmpt" Jul 07 07:05:13.827339 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.827298 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6572\" (UniqueName: \"kubernetes.io/projected/0b0f2bfd-706a-46ec-a92e-a02c5afe9b15-kube-api-access-f6572\") pod \"llmisvc-controller-manager-777c6bd7d5-gdc4p\" (UID: \"0b0f2bfd-706a-46ec-a92e-a02c5afe9b15\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p" Jul 07 07:05:13.828004 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.827979 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vws5l\" (UniqueName: \"kubernetes.io/projected/e1b1f08e-b37d-4cec-a942-87795b69dfec-kube-api-access-vws5l\") pod \"kserve-controller-manager-d6f4b65b8-twmpt\" (UID: \"e1b1f08e-b37d-4cec-a942-87795b69dfec\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-twmpt" Jul 07 07:05:13.878074 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.878018 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-twmpt" Jul 07 07:05:13.914534 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:13.914507 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:14.004771 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:14.004740 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-twmpt"] Jul 07 07:05:14.007681 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:05:14.007658 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1b1f08e_b37d_4cec_a942_87795b69dfec.slice/crio-8e32bdbe577fe5d590c4b7cdb42ff6fb340492e695bf849d93cb4cfca204e3de WatchSource:0}: Error finding container 8e32bdbe577fe5d590c4b7cdb42ff6fb340492e695bf849d93cb4cfca204e3de: Status 404 returned error can't find the container with id 8e32bdbe577fe5d590c4b7cdb42ff6fb340492e695bf849d93cb4cfca204e3de Jul 07 07:05:14.008973 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:14.008939 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:14.320034 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:14.319998 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b0f2bfd-706a-46ec-a92e-a02c5afe9b15-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gdc4p\" (UID: \"0b0f2bfd-706a-46ec-a92e-a02c5afe9b15\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p" Jul 07 07:05:14.322348 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:14.322310 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b0f2bfd-706a-46ec-a92e-a02c5afe9b15-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gdc4p\" (UID: \"0b0f2bfd-706a-46ec-a92e-a02c5afe9b15\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p" Jul 07 07:05:14.425967 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:14.425933 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:14.510974 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:14.510946 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p" Jul 07 07:05:14.556223 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:14.556195 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:14.640946 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:14.640923 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p"] Jul 07 07:05:14.642455 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:05:14.642431 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0b0f2bfd_706a_46ec_a92e_a02c5afe9b15.slice/crio-0d5a35e2f999243767cd30a0f36cc4f6345eeb9d742ab57ba24956ac1032bfd2 WatchSource:0}: Error finding container 0d5a35e2f999243767cd30a0f36cc4f6345eeb9d742ab57ba24956ac1032bfd2: Status 404 returned error can't find the container with id 0d5a35e2f999243767cd30a0f36cc4f6345eeb9d742ab57ba24956ac1032bfd2 Jul 07 07:05:14.644074 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:14.644046 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:14.650825 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:14.650801 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p" event={"ID":"0b0f2bfd-706a-46ec-a92e-a02c5afe9b15","Type":"ContainerStarted","Data":"0d5a35e2f999243767cd30a0f36cc4f6345eeb9d742ab57ba24956ac1032bfd2"} Jul 07 07:05:14.651883 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:14.651863 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-twmpt" event={"ID":"e1b1f08e-b37d-4cec-a942-87795b69dfec","Type":"ContainerStarted","Data":"8e32bdbe577fe5d590c4b7cdb42ff6fb340492e695bf849d93cb4cfca204e3de"} Jul 07 07:05:15.108711 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:15.108677 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25e409ee-a5df-4d7a-b2ce-815c1ac600d6" path="/var/lib/kubelet/pods/25e409ee-a5df-4d7a-b2ce-815c1ac600d6/volumes" Jul 07 07:05:17.669603 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:17.669568 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-twmpt" event={"ID":"e1b1f08e-b37d-4cec-a942-87795b69dfec","Type":"ContainerStarted","Data":"6e6517c7f57044ab67a2e1c8b6878ac91740cac36d63e9243158757b5f88c6de"} Jul 07 07:05:17.670061 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:17.669720 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-twmpt" Jul 07 07:05:17.716261 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:17.716211 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-twmpt" podStartSLOduration=1.905454687 podStartE2EDuration="4.716195348s" totalImagesPullingTime="2.810740661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:05:17.715142287 +0000 UTC m=+373.051108605" watchObservedRunningTime="2026-07-07 07:05:17.716195348 +0000 UTC m=+373.052161665" Jul 07 07:05:18.675204 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:18.675102 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p" event={"ID":"0b0f2bfd-706a-46ec-a92e-a02c5afe9b15","Type":"ContainerStarted","Data":"f8bfc007d788ded85b7b0bae7d1a9ec326d126b22c235529a4654ab9c07d202e"} Jul 07 07:05:18.701169 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:18.701128 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p" podStartSLOduration=2.043909235 podStartE2EDuration="5.701112354s" totalImagesPullingTime="3.657203119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:05:18.70053433 +0000 UTC m=+374.036500650" watchObservedRunningTime="2026-07-07 07:05:18.701112354 +0000 UTC m=+374.037078671" Jul 07 07:05:19.625552 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:19.625520 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2j7jp" Jul 07 07:05:19.627666 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:19.627646 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-gdsdv" Jul 07 07:05:19.680204 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:19.680163 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p" Jul 07 07:05:48.680281 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:48.680254 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-twmpt" Jul 07 07:05:50.686534 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:05:50.686502 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdc4p" Jul 07 07:06:02.079629 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:02.079598 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:02.181529 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:02.181495 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:02.257023 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:02.256992 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:03.009596 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:03.009564 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:03.198731 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:03.198699 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:31.077046 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:31.076957 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:34.659257 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:34.659208 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:37.800997 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:37.800960 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:39.203504 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:39.203474 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:39.460308 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:39.460232 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:40.089031 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:40.088998 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:42.481870 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:42.481838 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cqx5c"] Jul 07 07:06:42.483927 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:42.483910 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cqx5c" Jul 07 07:06:42.487607 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:42.487582 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 07:06:42.487776 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:42.487759 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bvj7h\"" Jul 07 07:06:42.487830 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:42.487790 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 07:06:42.496365 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:42.496342 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cqx5c"] Jul 07 07:06:42.519820 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:42.519791 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg5rz\" (UniqueName: \"kubernetes.io/projected/d5abeee0-d813-4ea1-819f-0016c04901f7-kube-api-access-fg5rz\") pod \"authorino-operator-75cc8d5cb7-cqx5c\" (UID: \"d5abeee0-d813-4ea1-819f-0016c04901f7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cqx5c" Jul 07 07:06:42.529872 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:42.529847 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:42.620671 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:42.620646 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg5rz\" (UniqueName: \"kubernetes.io/projected/d5abeee0-d813-4ea1-819f-0016c04901f7-kube-api-access-fg5rz\") pod \"authorino-operator-75cc8d5cb7-cqx5c\" (UID: \"d5abeee0-d813-4ea1-819f-0016c04901f7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cqx5c" Jul 07 07:06:42.640266 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:42.640246 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg5rz\" (UniqueName: \"kubernetes.io/projected/d5abeee0-d813-4ea1-819f-0016c04901f7-kube-api-access-fg5rz\") pod \"authorino-operator-75cc8d5cb7-cqx5c\" (UID: \"d5abeee0-d813-4ea1-819f-0016c04901f7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cqx5c" Jul 07 07:06:42.646159 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:42.646134 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:42.793186 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:42.793156 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cqx5c" Jul 07 07:06:42.926341 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:42.926177 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cqx5c"] Jul 07 07:06:42.929125 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:06:42.929095 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5abeee0_d813_4ea1_819f_0016c04901f7.slice/crio-a3b554f80ae268a063369fa1a49bb2b619bb74321e7b0d34f3e87343a88be2be WatchSource:0}: Error finding container a3b554f80ae268a063369fa1a49bb2b619bb74321e7b0d34f3e87343a88be2be: Status 404 returned error can't find the container with id a3b554f80ae268a063369fa1a49bb2b619bb74321e7b0d34f3e87343a88be2be Jul 07 07:06:43.023583 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:43.023552 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cqx5c" event={"ID":"d5abeee0-d813-4ea1-819f-0016c04901f7","Type":"ContainerStarted","Data":"a3b554f80ae268a063369fa1a49bb2b619bb74321e7b0d34f3e87343a88be2be"} Jul 07 07:06:43.459075 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:43.459043 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:45.319743 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:45.319708 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:45.356971 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:45.356939 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm"] Jul 07 07:06:45.360528 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:45.360509 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" Jul 07 07:06:45.365771 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:45.365751 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-zvpzq\"" Jul 07 07:06:45.389721 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:45.389660 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm"] Jul 07 07:06:45.443563 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:45.443533 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mf6s\" (UniqueName: \"kubernetes.io/projected/c015bd57-1362-43d2-95ca-308a805a6ddc-kube-api-access-5mf6s\") pod \"limitador-operator-controller-manager-859d7b78bd-phhvm\" (UID: \"c015bd57-1362-43d2-95ca-308a805a6ddc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" Jul 07 07:06:45.544423 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:45.544392 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mf6s\" (UniqueName: \"kubernetes.io/projected/c015bd57-1362-43d2-95ca-308a805a6ddc-kube-api-access-5mf6s\") pod \"limitador-operator-controller-manager-859d7b78bd-phhvm\" (UID: \"c015bd57-1362-43d2-95ca-308a805a6ddc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" Jul 07 07:06:45.553771 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:45.553749 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mf6s\" (UniqueName: \"kubernetes.io/projected/c015bd57-1362-43d2-95ca-308a805a6ddc-kube-api-access-5mf6s\") pod \"limitador-operator-controller-manager-859d7b78bd-phhvm\" (UID: \"c015bd57-1362-43d2-95ca-308a805a6ddc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" Jul 07 07:06:45.670248 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:45.670169 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" Jul 07 07:06:45.802275 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:45.802253 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm"] Jul 07 07:06:45.804561 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:06:45.804536 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc015bd57_1362_43d2_95ca_308a805a6ddc.slice/crio-870ec972958b819cdb435bea994382cc5f8d8f21a88b7e7ab5cbe8cf31ac7a4a WatchSource:0}: Error finding container 870ec972958b819cdb435bea994382cc5f8d8f21a88b7e7ab5cbe8cf31ac7a4a: Status 404 returned error can't find the container with id 870ec972958b819cdb435bea994382cc5f8d8f21a88b7e7ab5cbe8cf31ac7a4a Jul 07 07:06:46.039176 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:46.039139 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" event={"ID":"c015bd57-1362-43d2-95ca-308a805a6ddc","Type":"ContainerStarted","Data":"870ec972958b819cdb435bea994382cc5f8d8f21a88b7e7ab5cbe8cf31ac7a4a"} Jul 07 07:06:46.040531 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:46.040504 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cqx5c" event={"ID":"d5abeee0-d813-4ea1-819f-0016c04901f7","Type":"ContainerStarted","Data":"2aef100872971387b6da0e8d8402fb286ece2b5a88ff716934fffe30fc9c3f0e"} Jul 07 07:06:46.040667 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:46.040650 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cqx5c" Jul 07 07:06:46.063296 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:46.063247 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cqx5c" podStartSLOduration=1.845461728 podStartE2EDuration="4.063233634s" totalImagesPullingTime="2.217771906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:06:46.062777046 +0000 UTC m=+461.398743365" watchObservedRunningTime="2026-07-07 07:06:46.063233634 +0000 UTC m=+461.399200021" Jul 07 07:06:46.920053 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:46.920028 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:47.062504 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:47.062473 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:48.374765 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:48.374690 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:48.610278 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:48.610249 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:49.055407 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:49.055375 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" event={"ID":"c015bd57-1362-43d2-95ca-308a805a6ddc","Type":"ContainerStarted","Data":"b8e8cf924c624dd44ba34984ba9fbfe724ad013473404fd7e2837731c0910dcc"} Jul 07 07:06:49.055589 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:49.055488 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" Jul 07 07:06:49.081487 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:49.081443 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" podStartSLOduration=1.846166835 podStartE2EDuration="4.081430968s" totalImagesPullingTime="2.235264133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:06:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:06:49.078576649 +0000 UTC m=+464.414542965" watchObservedRunningTime="2026-07-07 07:06:49.081430968 +0000 UTC m=+464.417397284" Jul 07 07:06:50.673262 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:50.673225 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:50.772223 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:50.772189 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:57.048270 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:57.048234 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cqx5c" Jul 07 07:06:59.346945 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:59.346909 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm"] Jul 07 07:06:59.347401 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:59.347100 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" podUID="c015bd57-1362-43d2-95ca-308a805a6ddc" containerName="manager" containerID="cri-o://b8e8cf924c624dd44ba34984ba9fbfe724ad013473404fd7e2837731c0910dcc" gracePeriod=2 Jul 07 07:06:59.348652 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:59.348630 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" Jul 07 07:06:59.360058 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:59.360032 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm"] Jul 07 07:06:59.362883 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:06:59.362846 2573 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c015bd57-1362-43d2-95ca-308a805a6ddc\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-phhvm\": pods \"limitador-operator-controller-manager-859d7b78bd-phhvm\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" Jul 07 07:06:59.366504 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:06:59.366478 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-phhvm\" is forbidden: User \"system:node:ip-10-0-131-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-241.ec2.internal' and this object" podUID="c015bd57-1362-43d2-95ca-308a805a6ddc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" Jul 07 07:06:59.583252 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:59.583230 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" Jul 07 07:06:59.586058 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:06:59.586032 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-phhvm\" is forbidden: User \"system:node:ip-10-0-131-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-241.ec2.internal' and this object" podUID="c015bd57-1362-43d2-95ca-308a805a6ddc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" Jul 07 07:06:59.660691 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:59.660625 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mf6s\" (UniqueName: \"kubernetes.io/projected/c015bd57-1362-43d2-95ca-308a805a6ddc-kube-api-access-5mf6s\") pod \"c015bd57-1362-43d2-95ca-308a805a6ddc\" (UID: \"c015bd57-1362-43d2-95ca-308a805a6ddc\") " Jul 07 07:06:59.662723 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:59.662697 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c015bd57-1362-43d2-95ca-308a805a6ddc-kube-api-access-5mf6s" (OuterVolumeSpecName: "kube-api-access-5mf6s") pod "c015bd57-1362-43d2-95ca-308a805a6ddc" (UID: "c015bd57-1362-43d2-95ca-308a805a6ddc"). InnerVolumeSpecName "kube-api-access-5mf6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:06:59.761459 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:06:59.761430 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mf6s\" (UniqueName: \"kubernetes.io/projected/c015bd57-1362-43d2-95ca-308a805a6ddc-kube-api-access-5mf6s\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:07:00.104393 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:00.104351 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:00.105111 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:00.105090 2573 generic.go:356] "Generic (PLEG): container finished" podID="c015bd57-1362-43d2-95ca-308a805a6ddc" containerID="b8e8cf924c624dd44ba34984ba9fbfe724ad013473404fd7e2837731c0910dcc" exitCode=0 Jul 07 07:07:00.105174 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:00.105137 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" Jul 07 07:07:00.105174 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:00.105144 2573 scope.go:117] "RemoveContainer" containerID="b8e8cf924c624dd44ba34984ba9fbfe724ad013473404fd7e2837731c0910dcc" Jul 07 07:07:00.107653 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:07:00.107622 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-phhvm\" is forbidden: User \"system:node:ip-10-0-131-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-241.ec2.internal' and this object" podUID="c015bd57-1362-43d2-95ca-308a805a6ddc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" Jul 07 07:07:00.114122 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:00.114101 2573 scope.go:117] "RemoveContainer" containerID="b8e8cf924c624dd44ba34984ba9fbfe724ad013473404fd7e2837731c0910dcc" Jul 07 07:07:00.114471 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:07:00.114451 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8e8cf924c624dd44ba34984ba9fbfe724ad013473404fd7e2837731c0910dcc\": container with ID starting with b8e8cf924c624dd44ba34984ba9fbfe724ad013473404fd7e2837731c0910dcc not found: ID does not exist" containerID="b8e8cf924c624dd44ba34984ba9fbfe724ad013473404fd7e2837731c0910dcc" Jul 07 07:07:00.114550 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:00.114475 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8e8cf924c624dd44ba34984ba9fbfe724ad013473404fd7e2837731c0910dcc"} err="failed to get container status \"b8e8cf924c624dd44ba34984ba9fbfe724ad013473404fd7e2837731c0910dcc\": rpc error: code = NotFound desc = could not find container \"b8e8cf924c624dd44ba34984ba9fbfe724ad013473404fd7e2837731c0910dcc\": container with ID starting with b8e8cf924c624dd44ba34984ba9fbfe724ad013473404fd7e2837731c0910dcc not found: ID does not exist" Jul 07 07:07:00.116084 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:07:00.116062 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-phhvm\" is forbidden: User \"system:node:ip-10-0-131-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-241.ec2.internal' and this object" podUID="c015bd57-1362-43d2-95ca-308a805a6ddc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phhvm" Jul 07 07:07:00.204870 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:00.204838 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:01.107217 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:01.107187 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c015bd57-1362-43d2-95ca-308a805a6ddc" path="/var/lib/kubelet/pods/c015bd57-1362-43d2-95ca-308a805a6ddc/volumes" Jul 07 07:07:03.095850 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:03.095818 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:03.199456 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:03.199423 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:04.571965 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:04.571932 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:05.688043 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:05.688007 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:05.788367 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:05.788307 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:08.640800 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:08.640755 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:08.732402 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:08.732374 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:09.322809 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:09.322780 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:09.454705 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:09.454672 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:09.550101 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:09.550067 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:18.888598 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:18.888561 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:18.988952 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:18.988918 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:19.286523 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:19.286491 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:19.387695 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:19.387665 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:20.185335 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:20.185287 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:20.285990 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:20.285956 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:20.386673 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:20.386633 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:22.695586 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:22.695558 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:22.794899 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:22.794864 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:41.380650 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:41.380615 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:41.591925 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:41.591892 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:44.113717 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.113683 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fqzx6"] Jul 07 07:07:44.114088 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.114004 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c015bd57-1362-43d2-95ca-308a805a6ddc" containerName="manager" Jul 07 07:07:44.114088 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.114017 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c015bd57-1362-43d2-95ca-308a805a6ddc" containerName="manager" Jul 07 07:07:44.114088 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.114067 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c015bd57-1362-43d2-95ca-308a805a6ddc" containerName="manager" Jul 07 07:07:44.116864 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.116845 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" Jul 07 07:07:44.119423 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.119402 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 07:07:44.119553 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.119463 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dbd4w\"" Jul 07 07:07:44.122516 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.122495 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f45332fc-5ec1-4d08-b196-775ebd4e5639-config-file\") pod \"limitador-limitador-64c94fd64f-fqzx6\" (UID: \"f45332fc-5ec1-4d08-b196-775ebd4e5639\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" Jul 07 07:07:44.122596 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.122534 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whrmr\" (UniqueName: \"kubernetes.io/projected/f45332fc-5ec1-4d08-b196-775ebd4e5639-kube-api-access-whrmr\") pod \"limitador-limitador-64c94fd64f-fqzx6\" (UID: \"f45332fc-5ec1-4d08-b196-775ebd4e5639\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" Jul 07 07:07:44.126483 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.126460 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fqzx6"] Jul 07 07:07:44.177407 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.177373 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:44.206480 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.206453 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fqzx6"] Jul 07 07:07:44.223428 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.223404 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f45332fc-5ec1-4d08-b196-775ebd4e5639-config-file\") pod \"limitador-limitador-64c94fd64f-fqzx6\" (UID: \"f45332fc-5ec1-4d08-b196-775ebd4e5639\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" Jul 07 07:07:44.223524 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.223443 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whrmr\" (UniqueName: \"kubernetes.io/projected/f45332fc-5ec1-4d08-b196-775ebd4e5639-kube-api-access-whrmr\") pod \"limitador-limitador-64c94fd64f-fqzx6\" (UID: \"f45332fc-5ec1-4d08-b196-775ebd4e5639\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" Jul 07 07:07:44.224061 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.224044 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f45332fc-5ec1-4d08-b196-775ebd4e5639-config-file\") pod \"limitador-limitador-64c94fd64f-fqzx6\" (UID: \"f45332fc-5ec1-4d08-b196-775ebd4e5639\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" Jul 07 07:07:44.232942 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.232916 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whrmr\" (UniqueName: \"kubernetes.io/projected/f45332fc-5ec1-4d08-b196-775ebd4e5639-kube-api-access-whrmr\") pod \"limitador-limitador-64c94fd64f-fqzx6\" (UID: \"f45332fc-5ec1-4d08-b196-775ebd4e5639\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" Jul 07 07:07:44.280779 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.280755 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:44.427146 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.427049 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" Jul 07 07:07:44.556240 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.556215 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fqzx6"] Jul 07 07:07:44.558552 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:07:44.558524 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf45332fc_5ec1_4d08_b196_775ebd4e5639.slice/crio-eb622320b913987f502b5eaaa6e0e4e2540598655230fc8bf292ea797b8b6a98 WatchSource:0}: Error finding container eb622320b913987f502b5eaaa6e0e4e2540598655230fc8bf292ea797b8b6a98: Status 404 returned error can't find the container with id eb622320b913987f502b5eaaa6e0e4e2540598655230fc8bf292ea797b8b6a98 Jul 07 07:07:44.992950 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:44.992917 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:45.086032 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:45.086000 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:45.300754 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:45.300719 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" event={"ID":"f45332fc-5ec1-4d08-b196-775ebd4e5639","Type":"ContainerStarted","Data":"eb622320b913987f502b5eaaa6e0e4e2540598655230fc8bf292ea797b8b6a98"} Jul 07 07:07:48.318813 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:48.318782 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" event={"ID":"f45332fc-5ec1-4d08-b196-775ebd4e5639","Type":"ContainerStarted","Data":"4f665aa964729bd4f0122d55617261389e64dc8cfc79928baf863359cd2704ac"} Jul 07 07:07:48.319193 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:48.318917 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" Jul 07 07:07:48.337809 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:48.337746 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" podStartSLOduration=1.100029258 podStartE2EDuration="4.337730889s" totalImagesPullingTime="3.237701631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:07:48.336889699 +0000 UTC m=+523.672856030" watchObservedRunningTime="2026-07-07 07:07:48.337730889 +0000 UTC m=+523.673697207" Jul 07 07:07:59.002430 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:59.002340 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fqzx6"] Jul 07 07:07:59.002869 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:59.002646 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" podUID="f45332fc-5ec1-4d08-b196-775ebd4e5639" containerName="limitador" containerID="cri-o://4f665aa964729bd4f0122d55617261389e64dc8cfc79928baf863359cd2704ac" gracePeriod=30 Jul 07 07:07:59.003770 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:59.003417 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" Jul 07 07:07:59.553923 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:59.553902 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" Jul 07 07:07:59.650451 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:59.650386 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f45332fc-5ec1-4d08-b196-775ebd4e5639-config-file\") pod \"f45332fc-5ec1-4d08-b196-775ebd4e5639\" (UID: \"f45332fc-5ec1-4d08-b196-775ebd4e5639\") " Jul 07 07:07:59.650592 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:59.650457 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whrmr\" (UniqueName: \"kubernetes.io/projected/f45332fc-5ec1-4d08-b196-775ebd4e5639-kube-api-access-whrmr\") pod \"f45332fc-5ec1-4d08-b196-775ebd4e5639\" (UID: \"f45332fc-5ec1-4d08-b196-775ebd4e5639\") " Jul 07 07:07:59.650778 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:59.650755 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f45332fc-5ec1-4d08-b196-775ebd4e5639-config-file" (OuterVolumeSpecName: "config-file") pod "f45332fc-5ec1-4d08-b196-775ebd4e5639" (UID: "f45332fc-5ec1-4d08-b196-775ebd4e5639"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:07:59.652410 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:59.652388 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f45332fc-5ec1-4d08-b196-775ebd4e5639-kube-api-access-whrmr" (OuterVolumeSpecName: "kube-api-access-whrmr") pod "f45332fc-5ec1-4d08-b196-775ebd4e5639" (UID: "f45332fc-5ec1-4d08-b196-775ebd4e5639"). InnerVolumeSpecName "kube-api-access-whrmr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:07:59.751484 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:59.751443 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whrmr\" (UniqueName: \"kubernetes.io/projected/f45332fc-5ec1-4d08-b196-775ebd4e5639-kube-api-access-whrmr\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:07:59.751484 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:07:59.751480 2573 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f45332fc-5ec1-4d08-b196-775ebd4e5639-config-file\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:08:00.369277 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.369241 2573 generic.go:356] "Generic (PLEG): container finished" podID="f45332fc-5ec1-4d08-b196-775ebd4e5639" containerID="4f665aa964729bd4f0122d55617261389e64dc8cfc79928baf863359cd2704ac" exitCode=0 Jul 07 07:08:00.369691 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.369288 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" Jul 07 07:08:00.369691 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.369306 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" event={"ID":"f45332fc-5ec1-4d08-b196-775ebd4e5639","Type":"ContainerDied","Data":"4f665aa964729bd4f0122d55617261389e64dc8cfc79928baf863359cd2704ac"} Jul 07 07:08:00.369691 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.369381 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fqzx6" event={"ID":"f45332fc-5ec1-4d08-b196-775ebd4e5639","Type":"ContainerDied","Data":"eb622320b913987f502b5eaaa6e0e4e2540598655230fc8bf292ea797b8b6a98"} Jul 07 07:08:00.369691 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.369411 2573 scope.go:117] "RemoveContainer" containerID="4f665aa964729bd4f0122d55617261389e64dc8cfc79928baf863359cd2704ac" Jul 07 07:08:00.378242 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.378216 2573 scope.go:117] "RemoveContainer" containerID="4f665aa964729bd4f0122d55617261389e64dc8cfc79928baf863359cd2704ac" Jul 07 07:08:00.378550 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:08:00.378531 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f665aa964729bd4f0122d55617261389e64dc8cfc79928baf863359cd2704ac\": container with ID starting with 4f665aa964729bd4f0122d55617261389e64dc8cfc79928baf863359cd2704ac not found: ID does not exist" containerID="4f665aa964729bd4f0122d55617261389e64dc8cfc79928baf863359cd2704ac" Jul 07 07:08:00.378606 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.378560 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f665aa964729bd4f0122d55617261389e64dc8cfc79928baf863359cd2704ac"} err="failed to get container status \"4f665aa964729bd4f0122d55617261389e64dc8cfc79928baf863359cd2704ac\": rpc error: code = NotFound desc = could not find container \"4f665aa964729bd4f0122d55617261389e64dc8cfc79928baf863359cd2704ac\": container with ID starting with 4f665aa964729bd4f0122d55617261389e64dc8cfc79928baf863359cd2704ac not found: ID does not exist" Jul 07 07:08:00.391832 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.391809 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fqzx6"] Jul 07 07:08:00.395580 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.395558 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fqzx6"] Jul 07 07:08:00.544723 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.544699 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-7hlst"] Jul 07 07:08:00.545017 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.545001 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f45332fc-5ec1-4d08-b196-775ebd4e5639" containerName="limitador" Jul 07 07:08:00.545017 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.545016 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f45332fc-5ec1-4d08-b196-775ebd4e5639" containerName="limitador" Jul 07 07:08:00.545153 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.545071 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f45332fc-5ec1-4d08-b196-775ebd4e5639" containerName="limitador" Jul 07 07:08:00.549013 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.548997 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-7hlst" Jul 07 07:08:00.551456 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.551436 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 07 07:08:00.551560 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.551517 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ff8jg\"" Jul 07 07:08:00.558081 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.558057 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-7hlst"] Jul 07 07:08:00.658831 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.658760 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/832f39b3-d091-4a25-b80a-241d675b5029-data\") pod \"postgres-69fb4594cf-7hlst\" (UID: \"832f39b3-d091-4a25-b80a-241d675b5029\") " pod="opendatahub/postgres-69fb4594cf-7hlst" Jul 07 07:08:00.658831 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.658803 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6zll\" (UniqueName: \"kubernetes.io/projected/832f39b3-d091-4a25-b80a-241d675b5029-kube-api-access-t6zll\") pod \"postgres-69fb4594cf-7hlst\" (UID: \"832f39b3-d091-4a25-b80a-241d675b5029\") " pod="opendatahub/postgres-69fb4594cf-7hlst" Jul 07 07:08:00.707375 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.707347 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:00.759344 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.759303 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/832f39b3-d091-4a25-b80a-241d675b5029-data\") pod \"postgres-69fb4594cf-7hlst\" (UID: \"832f39b3-d091-4a25-b80a-241d675b5029\") " pod="opendatahub/postgres-69fb4594cf-7hlst" Jul 07 07:08:00.759470 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.759379 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6zll\" (UniqueName: \"kubernetes.io/projected/832f39b3-d091-4a25-b80a-241d675b5029-kube-api-access-t6zll\") pod \"postgres-69fb4594cf-7hlst\" (UID: \"832f39b3-d091-4a25-b80a-241d675b5029\") " pod="opendatahub/postgres-69fb4594cf-7hlst" Jul 07 07:08:00.759663 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.759646 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/832f39b3-d091-4a25-b80a-241d675b5029-data\") pod \"postgres-69fb4594cf-7hlst\" (UID: \"832f39b3-d091-4a25-b80a-241d675b5029\") " pod="opendatahub/postgres-69fb4594cf-7hlst" Jul 07 07:08:00.769811 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.769785 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6zll\" (UniqueName: \"kubernetes.io/projected/832f39b3-d091-4a25-b80a-241d675b5029-kube-api-access-t6zll\") pod \"postgres-69fb4594cf-7hlst\" (UID: \"832f39b3-d091-4a25-b80a-241d675b5029\") " pod="opendatahub/postgres-69fb4594cf-7hlst" Jul 07 07:08:00.859465 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.859435 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-7hlst" Jul 07 07:08:00.986807 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:00.986772 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-7hlst"] Jul 07 07:08:00.989645 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:08:00.989606 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod832f39b3_d091_4a25_b80a_241d675b5029.slice/crio-9790c06becc21d9f7e9c95d72dcb183d5d17b184aa0836e915af114b33186d8d WatchSource:0}: Error finding container 9790c06becc21d9f7e9c95d72dcb183d5d17b184aa0836e915af114b33186d8d: Status 404 returned error can't find the container with id 9790c06becc21d9f7e9c95d72dcb183d5d17b184aa0836e915af114b33186d8d Jul 07 07:08:01.107118 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:01.107090 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f45332fc-5ec1-4d08-b196-775ebd4e5639" path="/var/lib/kubelet/pods/f45332fc-5ec1-4d08-b196-775ebd4e5639/volumes" Jul 07 07:08:01.374646 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:01.374612 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-7hlst" event={"ID":"832f39b3-d091-4a25-b80a-241d675b5029","Type":"ContainerStarted","Data":"9790c06becc21d9f7e9c95d72dcb183d5d17b184aa0836e915af114b33186d8d"} Jul 07 07:08:05.914199 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:05.914179 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 07 07:08:06.357902 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:06.357870 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:06.403154 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:06.403121 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-7hlst" event={"ID":"832f39b3-d091-4a25-b80a-241d675b5029","Type":"ContainerStarted","Data":"f8ada94b02a85f10c578862651d7c1e0e9bbc84fb1fb9be0c27030e5f1004ec2"} Jul 07 07:08:06.403305 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:06.403249 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-7hlst" Jul 07 07:08:06.421854 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:06.421809 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-7hlst" podStartSLOduration=1.501119206 podStartE2EDuration="6.421794241s" totalImagesPullingTime="4.920675035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:08:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:08:06.420242474 +0000 UTC m=+541.756208813" watchObservedRunningTime="2026-07-07 07:08:06.421794241 +0000 UTC m=+541.757760558" Jul 07 07:08:12.436884 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:12.436855 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-7hlst" Jul 07 07:08:12.754314 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:12.754285 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:12.998034 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:12.998002 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-256vb"] Jul 07 07:08:13.002895 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:13.002872 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-256vb" Jul 07 07:08:13.004850 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:13.004742 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:13.005236 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:13.005218 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9f64d\"" Jul 07 07:08:13.011768 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:13.011744 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-256vb"] Jul 07 07:08:13.072076 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:13.072036 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx2jc\" (UniqueName: \"kubernetes.io/projected/1c435b3e-03d6-4e9e-9d18-ffdc9245ca77-kube-api-access-hx2jc\") pod \"authorino-5d4b67c446-256vb\" (UID: \"1c435b3e-03d6-4e9e-9d18-ffdc9245ca77\") " pod="kuadrant-system/authorino-5d4b67c446-256vb" Jul 07 07:08:13.172873 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:13.172834 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx2jc\" (UniqueName: \"kubernetes.io/projected/1c435b3e-03d6-4e9e-9d18-ffdc9245ca77-kube-api-access-hx2jc\") pod \"authorino-5d4b67c446-256vb\" (UID: \"1c435b3e-03d6-4e9e-9d18-ffdc9245ca77\") " pod="kuadrant-system/authorino-5d4b67c446-256vb" Jul 07 07:08:13.182607 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:13.182570 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx2jc\" (UniqueName: \"kubernetes.io/projected/1c435b3e-03d6-4e9e-9d18-ffdc9245ca77-kube-api-access-hx2jc\") pod \"authorino-5d4b67c446-256vb\" (UID: \"1c435b3e-03d6-4e9e-9d18-ffdc9245ca77\") " pod="kuadrant-system/authorino-5d4b67c446-256vb" Jul 07 07:08:13.251121 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:13.251082 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-256vb"] Jul 07 07:08:13.251346 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:13.251317 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-256vb" Jul 07 07:08:13.379974 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:13.379940 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-256vb"] Jul 07 07:08:13.383028 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:08:13.383003 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c435b3e_03d6_4e9e_9d18_ffdc9245ca77.slice/crio-e227b825179158054adf35eb0e8feca4d79a36c302bb279262b2a562dbbce7fe WatchSource:0}: Error finding container e227b825179158054adf35eb0e8feca4d79a36c302bb279262b2a562dbbce7fe: Status 404 returned error can't find the container with id e227b825179158054adf35eb0e8feca4d79a36c302bb279262b2a562dbbce7fe Jul 07 07:08:13.439164 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:13.439131 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-256vb" event={"ID":"1c435b3e-03d6-4e9e-9d18-ffdc9245ca77","Type":"ContainerStarted","Data":"e227b825179158054adf35eb0e8feca4d79a36c302bb279262b2a562dbbce7fe"} Jul 07 07:08:18.463075 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:18.463039 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-256vb" event={"ID":"1c435b3e-03d6-4e9e-9d18-ffdc9245ca77","Type":"ContainerStarted","Data":"b1e8a8dcfd727dc8e7f07a1994236372da9ea058a824b200edeff4b3b9fe0f07"} Jul 07 07:08:18.463537 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:18.463148 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-256vb" podUID="1c435b3e-03d6-4e9e-9d18-ffdc9245ca77" containerName="authorino" containerID="cri-o://b1e8a8dcfd727dc8e7f07a1994236372da9ea058a824b200edeff4b3b9fe0f07" gracePeriod=30 Jul 07 07:08:18.479141 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:18.479094 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-256vb" podStartSLOduration=2.099726682 podStartE2EDuration="6.479078314s" totalImagesPullingTime="4.379351632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:08:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:08:18.478890242 +0000 UTC m=+553.814856582" watchObservedRunningTime="2026-07-07 07:08:18.479078314 +0000 UTC m=+553.815044633" Jul 07 07:08:18.721104 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:18.721053 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-256vb" Jul 07 07:08:18.819605 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:18.819569 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx2jc\" (UniqueName: \"kubernetes.io/projected/1c435b3e-03d6-4e9e-9d18-ffdc9245ca77-kube-api-access-hx2jc\") pod \"1c435b3e-03d6-4e9e-9d18-ffdc9245ca77\" (UID: \"1c435b3e-03d6-4e9e-9d18-ffdc9245ca77\") " Jul 07 07:08:18.821539 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:18.821511 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c435b3e-03d6-4e9e-9d18-ffdc9245ca77-kube-api-access-hx2jc" (OuterVolumeSpecName: "kube-api-access-hx2jc") pod "1c435b3e-03d6-4e9e-9d18-ffdc9245ca77" (UID: "1c435b3e-03d6-4e9e-9d18-ffdc9245ca77"). InnerVolumeSpecName "kube-api-access-hx2jc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:08:18.921095 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:18.921058 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx2jc\" (UniqueName: \"kubernetes.io/projected/1c435b3e-03d6-4e9e-9d18-ffdc9245ca77-kube-api-access-hx2jc\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:08:19.468178 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.468102 2573 generic.go:356] "Generic (PLEG): container finished" podID="1c435b3e-03d6-4e9e-9d18-ffdc9245ca77" containerID="b1e8a8dcfd727dc8e7f07a1994236372da9ea058a824b200edeff4b3b9fe0f07" exitCode=0 Jul 07 07:08:19.468178 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.468143 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-256vb" Jul 07 07:08:19.468657 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.468146 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-256vb" event={"ID":"1c435b3e-03d6-4e9e-9d18-ffdc9245ca77","Type":"ContainerDied","Data":"b1e8a8dcfd727dc8e7f07a1994236372da9ea058a824b200edeff4b3b9fe0f07"} Jul 07 07:08:19.468657 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.468262 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-256vb" event={"ID":"1c435b3e-03d6-4e9e-9d18-ffdc9245ca77","Type":"ContainerDied","Data":"e227b825179158054adf35eb0e8feca4d79a36c302bb279262b2a562dbbce7fe"} Jul 07 07:08:19.468657 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.468291 2573 scope.go:117] "RemoveContainer" containerID="b1e8a8dcfd727dc8e7f07a1994236372da9ea058a824b200edeff4b3b9fe0f07" Jul 07 07:08:19.476289 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.476271 2573 scope.go:117] "RemoveContainer" containerID="b1e8a8dcfd727dc8e7f07a1994236372da9ea058a824b200edeff4b3b9fe0f07" Jul 07 07:08:19.476587 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:08:19.476568 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1e8a8dcfd727dc8e7f07a1994236372da9ea058a824b200edeff4b3b9fe0f07\": container with ID starting with b1e8a8dcfd727dc8e7f07a1994236372da9ea058a824b200edeff4b3b9fe0f07 not found: ID does not exist" containerID="b1e8a8dcfd727dc8e7f07a1994236372da9ea058a824b200edeff4b3b9fe0f07" Jul 07 07:08:19.476645 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.476596 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1e8a8dcfd727dc8e7f07a1994236372da9ea058a824b200edeff4b3b9fe0f07"} err="failed to get container status \"b1e8a8dcfd727dc8e7f07a1994236372da9ea058a824b200edeff4b3b9fe0f07\": rpc error: code = NotFound desc = could not find container \"b1e8a8dcfd727dc8e7f07a1994236372da9ea058a824b200edeff4b3b9fe0f07\": container with ID starting with b1e8a8dcfd727dc8e7f07a1994236372da9ea058a824b200edeff4b3b9fe0f07 not found: ID does not exist" Jul 07 07:08:19.485580 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.485557 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-256vb"] Jul 07 07:08:19.489941 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.489919 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-256vb"] Jul 07 07:08:19.764649 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.764616 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:19.768692 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.768667 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-9d6b65dd8-s7cnn"] Jul 07 07:08:19.769008 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.768996 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c435b3e-03d6-4e9e-9d18-ffdc9245ca77" containerName="authorino" Jul 07 07:08:19.769054 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.769012 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c435b3e-03d6-4e9e-9d18-ffdc9245ca77" containerName="authorino" Jul 07 07:08:19.769105 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.769092 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c435b3e-03d6-4e9e-9d18-ffdc9245ca77" containerName="authorino" Jul 07 07:08:19.771516 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.771499 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" Jul 07 07:08:19.774288 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.774262 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 07:08:19.774416 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.774398 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 07:08:19.774486 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.774402 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rdpdd\"" Jul 07 07:08:19.784198 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.784168 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9d6b65dd8-s7cnn"] Jul 07 07:08:19.859371 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.859344 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:19.929643 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.929608 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/533a620b-3f9f-413c-a94b-0d69de85df56-webhook-cert\") pod \"maas-controller-9d6b65dd8-s7cnn\" (UID: \"533a620b-3f9f-413c-a94b-0d69de85df56\") " pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" Jul 07 07:08:19.929795 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.929658 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvpgh\" (UniqueName: \"kubernetes.io/projected/533a620b-3f9f-413c-a94b-0d69de85df56-kube-api-access-xvpgh\") pod \"maas-controller-9d6b65dd8-s7cnn\" (UID: \"533a620b-3f9f-413c-a94b-0d69de85df56\") " pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" Jul 07 07:08:19.949471 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.949446 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:19.953177 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.953153 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c8d7dbb94-r4mw8"] Jul 07 07:08:19.955729 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.955710 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" Jul 07 07:08:19.965579 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:19.965555 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c8d7dbb94-r4mw8"] Jul 07 07:08:20.030510 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:20.030426 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/533a620b-3f9f-413c-a94b-0d69de85df56-webhook-cert\") pod \"maas-controller-9d6b65dd8-s7cnn\" (UID: \"533a620b-3f9f-413c-a94b-0d69de85df56\") " pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" Jul 07 07:08:20.030510 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:20.030468 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvpgh\" (UniqueName: \"kubernetes.io/projected/533a620b-3f9f-413c-a94b-0d69de85df56-kube-api-access-xvpgh\") pod \"maas-controller-9d6b65dd8-s7cnn\" (UID: \"533a620b-3f9f-413c-a94b-0d69de85df56\") " pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" Jul 07 07:08:20.032889 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:20.032869 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/533a620b-3f9f-413c-a94b-0d69de85df56-webhook-cert\") pod \"maas-controller-9d6b65dd8-s7cnn\" (UID: \"533a620b-3f9f-413c-a94b-0d69de85df56\") " pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" Jul 07 07:08:20.039584 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:20.039558 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvpgh\" (UniqueName: \"kubernetes.io/projected/533a620b-3f9f-413c-a94b-0d69de85df56-kube-api-access-xvpgh\") pod \"maas-controller-9d6b65dd8-s7cnn\" (UID: \"533a620b-3f9f-413c-a94b-0d69de85df56\") " pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" Jul 07 07:08:20.084282 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:20.084260 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" Jul 07 07:08:20.131657 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:20.131626 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c6f195f-595c-4ce1-8a0e-6f1aed0f1522-webhook-cert\") pod \"maas-controller-6c8d7dbb94-r4mw8\" (UID: \"6c6f195f-595c-4ce1-8a0e-6f1aed0f1522\") " pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" Jul 07 07:08:20.131781 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:20.131699 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9q2w\" (UniqueName: \"kubernetes.io/projected/6c6f195f-595c-4ce1-8a0e-6f1aed0f1522-kube-api-access-h9q2w\") pod \"maas-controller-6c8d7dbb94-r4mw8\" (UID: \"6c6f195f-595c-4ce1-8a0e-6f1aed0f1522\") " pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" Jul 07 07:08:20.209069 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:20.209039 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9d6b65dd8-s7cnn"] Jul 07 07:08:20.212007 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:08:20.211980 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod533a620b_3f9f_413c_a94b_0d69de85df56.slice/crio-cf6fdd06a8600747edba09fb1af911c488b43e6d3ec3ca2a732131548d41c2bb WatchSource:0}: Error finding container cf6fdd06a8600747edba09fb1af911c488b43e6d3ec3ca2a732131548d41c2bb: Status 404 returned error can't find the container with id cf6fdd06a8600747edba09fb1af911c488b43e6d3ec3ca2a732131548d41c2bb Jul 07 07:08:20.232648 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:20.232620 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c6f195f-595c-4ce1-8a0e-6f1aed0f1522-webhook-cert\") pod \"maas-controller-6c8d7dbb94-r4mw8\" (UID: \"6c6f195f-595c-4ce1-8a0e-6f1aed0f1522\") " pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" Jul 07 07:08:20.232757 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:20.232656 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9q2w\" (UniqueName: \"kubernetes.io/projected/6c6f195f-595c-4ce1-8a0e-6f1aed0f1522-kube-api-access-h9q2w\") pod \"maas-controller-6c8d7dbb94-r4mw8\" (UID: \"6c6f195f-595c-4ce1-8a0e-6f1aed0f1522\") " pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" Jul 07 07:08:20.234856 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:20.234837 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c6f195f-595c-4ce1-8a0e-6f1aed0f1522-webhook-cert\") pod \"maas-controller-6c8d7dbb94-r4mw8\" (UID: \"6c6f195f-595c-4ce1-8a0e-6f1aed0f1522\") " pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" Jul 07 07:08:20.241650 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:20.241632 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9q2w\" (UniqueName: \"kubernetes.io/projected/6c6f195f-595c-4ce1-8a0e-6f1aed0f1522-kube-api-access-h9q2w\") pod \"maas-controller-6c8d7dbb94-r4mw8\" (UID: \"6c6f195f-595c-4ce1-8a0e-6f1aed0f1522\") " pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" Jul 07 07:08:20.265506 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:20.265489 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" Jul 07 07:08:20.391499 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:20.391458 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c8d7dbb94-r4mw8"] Jul 07 07:08:20.395105 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:08:20.395082 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c6f195f_595c_4ce1_8a0e_6f1aed0f1522.slice/crio-8c9d93a78929d57cf29004965a3b9c073409235a0d6c2f963346de7b89b692eb WatchSource:0}: Error finding container 8c9d93a78929d57cf29004965a3b9c073409235a0d6c2f963346de7b89b692eb: Status 404 returned error can't find the container with id 8c9d93a78929d57cf29004965a3b9c073409235a0d6c2f963346de7b89b692eb Jul 07 07:08:20.473519 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:20.473491 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" event={"ID":"6c6f195f-595c-4ce1-8a0e-6f1aed0f1522","Type":"ContainerStarted","Data":"8c9d93a78929d57cf29004965a3b9c073409235a0d6c2f963346de7b89b692eb"} Jul 07 07:08:20.475205 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:20.475184 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" event={"ID":"533a620b-3f9f-413c-a94b-0d69de85df56","Type":"ContainerStarted","Data":"cf6fdd06a8600747edba09fb1af911c488b43e6d3ec3ca2a732131548d41c2bb"} Jul 07 07:08:21.111663 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:21.111623 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c435b3e-03d6-4e9e-9d18-ffdc9245ca77" path="/var/lib/kubelet/pods/1c435b3e-03d6-4e9e-9d18-ffdc9245ca77/volumes" Jul 07 07:08:23.121977 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:23.121938 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:23.222182 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:23.222098 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:23.493091 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:23.492995 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" event={"ID":"533a620b-3f9f-413c-a94b-0d69de85df56","Type":"ContainerStarted","Data":"03caaf2dc09412be0204bf7438d3963a7938d2258a22fcf746523527baad28bf"} Jul 07 07:08:23.493273 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:23.493107 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" Jul 07 07:08:23.494453 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:23.494432 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" event={"ID":"6c6f195f-595c-4ce1-8a0e-6f1aed0f1522","Type":"ContainerStarted","Data":"90d010498971500bde994b07dee0e84d69994efa036f4de2ff256c7a38185ae7"} Jul 07 07:08:23.494573 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:23.494563 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" Jul 07 07:08:23.513495 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:23.513456 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" podStartSLOduration=1.8107573179999998 podStartE2EDuration="4.513443883s" totalImagesPullingTime="2.702686565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:08:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:08:23.512175549 +0000 UTC m=+558.848141899" watchObservedRunningTime="2026-07-07 07:08:23.513443883 +0000 UTC m=+558.849410200" Jul 07 07:08:23.532966 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:23.532927 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" podStartSLOduration=2.020521756 podStartE2EDuration="4.532915104s" totalImagesPullingTime="2.512393348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:08:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:08:23.531829511 +0000 UTC m=+558.867795840" watchObservedRunningTime="2026-07-07 07:08:23.532915104 +0000 UTC m=+558.868881446" Jul 07 07:08:34.504149 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:34.504116 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" Jul 07 07:08:34.504583 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:34.504268 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" Jul 07 07:08:34.553084 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:34.553051 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9d6b65dd8-s7cnn"] Jul 07 07:08:34.553358 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:34.553294 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" podUID="533a620b-3f9f-413c-a94b-0d69de85df56" containerName="manager" containerID="cri-o://03caaf2dc09412be0204bf7438d3963a7938d2258a22fcf746523527baad28bf" gracePeriod=30 Jul 07 07:08:34.800152 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:34.800126 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" Jul 07 07:08:34.850222 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:34.850193 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/533a620b-3f9f-413c-a94b-0d69de85df56-webhook-cert\") pod \"533a620b-3f9f-413c-a94b-0d69de85df56\" (UID: \"533a620b-3f9f-413c-a94b-0d69de85df56\") " Jul 07 07:08:34.850397 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:34.850230 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvpgh\" (UniqueName: \"kubernetes.io/projected/533a620b-3f9f-413c-a94b-0d69de85df56-kube-api-access-xvpgh\") pod \"533a620b-3f9f-413c-a94b-0d69de85df56\" (UID: \"533a620b-3f9f-413c-a94b-0d69de85df56\") " Jul 07 07:08:34.852502 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:34.852471 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/533a620b-3f9f-413c-a94b-0d69de85df56-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "533a620b-3f9f-413c-a94b-0d69de85df56" (UID: "533a620b-3f9f-413c-a94b-0d69de85df56"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:08:34.852502 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:34.852482 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/533a620b-3f9f-413c-a94b-0d69de85df56-kube-api-access-xvpgh" (OuterVolumeSpecName: "kube-api-access-xvpgh") pod "533a620b-3f9f-413c-a94b-0d69de85df56" (UID: "533a620b-3f9f-413c-a94b-0d69de85df56"). InnerVolumeSpecName "kube-api-access-xvpgh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:08:34.951249 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:34.951208 2573 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/533a620b-3f9f-413c-a94b-0d69de85df56-webhook-cert\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:08:34.951249 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:34.951243 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvpgh\" (UniqueName: \"kubernetes.io/projected/533a620b-3f9f-413c-a94b-0d69de85df56-kube-api-access-xvpgh\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:08:35.547387 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:35.547356 2573 generic.go:356] "Generic (PLEG): container finished" podID="533a620b-3f9f-413c-a94b-0d69de85df56" containerID="03caaf2dc09412be0204bf7438d3963a7938d2258a22fcf746523527baad28bf" exitCode=0 Jul 07 07:08:35.547821 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:35.547398 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" Jul 07 07:08:35.547821 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:35.547423 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" event={"ID":"533a620b-3f9f-413c-a94b-0d69de85df56","Type":"ContainerDied","Data":"03caaf2dc09412be0204bf7438d3963a7938d2258a22fcf746523527baad28bf"} Jul 07 07:08:35.547821 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:35.547463 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9d6b65dd8-s7cnn" event={"ID":"533a620b-3f9f-413c-a94b-0d69de85df56","Type":"ContainerDied","Data":"cf6fdd06a8600747edba09fb1af911c488b43e6d3ec3ca2a732131548d41c2bb"} Jul 07 07:08:35.547821 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:35.547486 2573 scope.go:117] "RemoveContainer" containerID="03caaf2dc09412be0204bf7438d3963a7938d2258a22fcf746523527baad28bf" Jul 07 07:08:35.555936 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:35.555769 2573 scope.go:117] "RemoveContainer" containerID="03caaf2dc09412be0204bf7438d3963a7938d2258a22fcf746523527baad28bf" Jul 07 07:08:35.556049 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:08:35.556030 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03caaf2dc09412be0204bf7438d3963a7938d2258a22fcf746523527baad28bf\": container with ID starting with 03caaf2dc09412be0204bf7438d3963a7938d2258a22fcf746523527baad28bf not found: ID does not exist" containerID="03caaf2dc09412be0204bf7438d3963a7938d2258a22fcf746523527baad28bf" Jul 07 07:08:35.556092 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:35.556057 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03caaf2dc09412be0204bf7438d3963a7938d2258a22fcf746523527baad28bf"} err="failed to get container status \"03caaf2dc09412be0204bf7438d3963a7938d2258a22fcf746523527baad28bf\": rpc error: code = NotFound desc = could not find container \"03caaf2dc09412be0204bf7438d3963a7938d2258a22fcf746523527baad28bf\": container with ID starting with 03caaf2dc09412be0204bf7438d3963a7938d2258a22fcf746523527baad28bf not found: ID does not exist" Jul 07 07:08:35.566344 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:35.566302 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9d6b65dd8-s7cnn"] Jul 07 07:08:35.571098 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:35.571073 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-9d6b65dd8-s7cnn"] Jul 07 07:08:36.096528 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:36.096494 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:36.196959 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:36.196927 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:37.107439 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:37.107409 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="533a620b-3f9f-413c-a94b-0d69de85df56" path="/var/lib/kubelet/pods/533a620b-3f9f-413c-a94b-0d69de85df56/volumes" Jul 07 07:08:54.359126 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:54.359095 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:54.761920 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:54.761891 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:54.814980 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:54.814946 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-c87c98569-w4sbj"] Jul 07 07:08:54.815271 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:54.815258 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="533a620b-3f9f-413c-a94b-0d69de85df56" containerName="manager" Jul 07 07:08:54.815358 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:54.815272 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="533a620b-3f9f-413c-a94b-0d69de85df56" containerName="manager" Jul 07 07:08:54.815416 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:54.815361 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="533a620b-3f9f-413c-a94b-0d69de85df56" containerName="manager" Jul 07 07:08:54.819665 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:54.819644 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-c87c98569-w4sbj" Jul 07 07:08:54.822269 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:54.822251 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 07 07:08:54.822364 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:54.822250 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-z6nck\"" Jul 07 07:08:54.831552 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:54.831531 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-c87c98569-w4sbj"] Jul 07 07:08:54.907844 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:54.907813 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmdp4\" (UniqueName: \"kubernetes.io/projected/557b6ddd-adcf-4000-9175-a90c34dd4a37-kube-api-access-gmdp4\") pod \"maas-api-c87c98569-w4sbj\" (UID: \"557b6ddd-adcf-4000-9175-a90c34dd4a37\") " pod="opendatahub/maas-api-c87c98569-w4sbj" Jul 07 07:08:54.907844 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:54.907847 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/557b6ddd-adcf-4000-9175-a90c34dd4a37-maas-api-tls\") pod \"maas-api-c87c98569-w4sbj\" (UID: \"557b6ddd-adcf-4000-9175-a90c34dd4a37\") " pod="opendatahub/maas-api-c87c98569-w4sbj" Jul 07 07:08:54.908030 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:54.907933 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/557b6ddd-adcf-4000-9175-a90c34dd4a37-openshift-service-ca\") pod \"maas-api-c87c98569-w4sbj\" (UID: \"557b6ddd-adcf-4000-9175-a90c34dd4a37\") " pod="opendatahub/maas-api-c87c98569-w4sbj" Jul 07 07:08:55.008493 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.008442 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:55.008493 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.008463 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/557b6ddd-adcf-4000-9175-a90c34dd4a37-openshift-service-ca\") pod \"maas-api-c87c98569-w4sbj\" (UID: \"557b6ddd-adcf-4000-9175-a90c34dd4a37\") " pod="opendatahub/maas-api-c87c98569-w4sbj" Jul 07 07:08:55.008739 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.008540 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmdp4\" (UniqueName: \"kubernetes.io/projected/557b6ddd-adcf-4000-9175-a90c34dd4a37-kube-api-access-gmdp4\") pod \"maas-api-c87c98569-w4sbj\" (UID: \"557b6ddd-adcf-4000-9175-a90c34dd4a37\") " pod="opendatahub/maas-api-c87c98569-w4sbj" Jul 07 07:08:55.008739 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.008569 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/557b6ddd-adcf-4000-9175-a90c34dd4a37-maas-api-tls\") pod \"maas-api-c87c98569-w4sbj\" (UID: \"557b6ddd-adcf-4000-9175-a90c34dd4a37\") " pod="opendatahub/maas-api-c87c98569-w4sbj" Jul 07 07:08:55.009211 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.009181 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/557b6ddd-adcf-4000-9175-a90c34dd4a37-openshift-service-ca\") pod \"maas-api-c87c98569-w4sbj\" (UID: \"557b6ddd-adcf-4000-9175-a90c34dd4a37\") " pod="opendatahub/maas-api-c87c98569-w4sbj" Jul 07 07:08:55.010998 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.010978 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/557b6ddd-adcf-4000-9175-a90c34dd4a37-maas-api-tls\") pod \"maas-api-c87c98569-w4sbj\" (UID: \"557b6ddd-adcf-4000-9175-a90c34dd4a37\") " pod="opendatahub/maas-api-c87c98569-w4sbj" Jul 07 07:08:55.018348 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.018263 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmdp4\" (UniqueName: \"kubernetes.io/projected/557b6ddd-adcf-4000-9175-a90c34dd4a37-kube-api-access-gmdp4\") pod \"maas-api-c87c98569-w4sbj\" (UID: \"557b6ddd-adcf-4000-9175-a90c34dd4a37\") " pod="opendatahub/maas-api-c87c98569-w4sbj" Jul 07 07:08:55.129791 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.129765 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-c87c98569-w4sbj" Jul 07 07:08:55.269289 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.269258 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-c87c98569-w4sbj"] Jul 07 07:08:55.271706 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:08:55.271675 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod557b6ddd_adcf_4000_9175_a90c34dd4a37.slice/crio-674e13ca49d9a9cfb4a8fbb78cc8d7353c455c86a61d1ff4e27f25ca2adf4047 WatchSource:0}: Error finding container 674e13ca49d9a9cfb4a8fbb78cc8d7353c455c86a61d1ff4e27f25ca2adf4047: Status 404 returned error can't find the container with id 674e13ca49d9a9cfb4a8fbb78cc8d7353c455c86a61d1ff4e27f25ca2adf4047 Jul 07 07:08:55.301946 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.301913 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:55.411953 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.411921 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-44lzc"] Jul 07 07:08:55.414647 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.414623 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:55.414878 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.414860 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-44lzc" Jul 07 07:08:55.417165 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.417142 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 07 07:08:55.417280 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.417264 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-hrlvd\"" Jul 07 07:08:55.425971 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.425948 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-44lzc"] Jul 07 07:08:55.435368 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.435347 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-kk64t"] Jul 07 07:08:55.437609 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.437591 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-kk64t" Jul 07 07:08:55.449436 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.449414 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-kk64t"] Jul 07 07:08:55.512897 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.512864 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c1d716f-4fcd-4bf9-bda4-19db7059dc37-plugins-config-volume\") pod \"payload-processing-58f86c7798-kk64t\" (UID: \"5c1d716f-4fcd-4bf9-bda4-19db7059dc37\") " pod="openshift-ingress/payload-processing-58f86c7798-kk64t" Jul 07 07:08:55.512897 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.512901 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltj7q\" (UniqueName: \"kubernetes.io/projected/5c1d716f-4fcd-4bf9-bda4-19db7059dc37-kube-api-access-ltj7q\") pod \"payload-processing-58f86c7798-kk64t\" (UID: \"5c1d716f-4fcd-4bf9-bda4-19db7059dc37\") " pod="openshift-ingress/payload-processing-58f86c7798-kk64t" Jul 07 07:08:55.513097 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.512931 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cb8b73d-18bb-4ca8-b2af-a00c971c928d-plugins-config-volume\") pod \"payload-pre-processing-7964457696-44lzc\" (UID: \"2cb8b73d-18bb-4ca8-b2af-a00c971c928d\") " pod="openshift-ingress/payload-pre-processing-7964457696-44lzc" Jul 07 07:08:55.513097 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.513025 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dfdx\" (UniqueName: \"kubernetes.io/projected/2cb8b73d-18bb-4ca8-b2af-a00c971c928d-kube-api-access-5dfdx\") pod \"payload-pre-processing-7964457696-44lzc\" (UID: \"2cb8b73d-18bb-4ca8-b2af-a00c971c928d\") " pod="openshift-ingress/payload-pre-processing-7964457696-44lzc" Jul 07 07:08:55.614467 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.614382 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltj7q\" (UniqueName: \"kubernetes.io/projected/5c1d716f-4fcd-4bf9-bda4-19db7059dc37-kube-api-access-ltj7q\") pod \"payload-processing-58f86c7798-kk64t\" (UID: \"5c1d716f-4fcd-4bf9-bda4-19db7059dc37\") " pod="openshift-ingress/payload-processing-58f86c7798-kk64t" Jul 07 07:08:55.614467 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.614433 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cb8b73d-18bb-4ca8-b2af-a00c971c928d-plugins-config-volume\") pod \"payload-pre-processing-7964457696-44lzc\" (UID: \"2cb8b73d-18bb-4ca8-b2af-a00c971c928d\") " pod="openshift-ingress/payload-pre-processing-7964457696-44lzc" Jul 07 07:08:55.614467 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.614466 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dfdx\" (UniqueName: \"kubernetes.io/projected/2cb8b73d-18bb-4ca8-b2af-a00c971c928d-kube-api-access-5dfdx\") pod \"payload-pre-processing-7964457696-44lzc\" (UID: \"2cb8b73d-18bb-4ca8-b2af-a00c971c928d\") " pod="openshift-ingress/payload-pre-processing-7964457696-44lzc" Jul 07 07:08:55.614687 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.614513 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c1d716f-4fcd-4bf9-bda4-19db7059dc37-plugins-config-volume\") pod \"payload-processing-58f86c7798-kk64t\" (UID: \"5c1d716f-4fcd-4bf9-bda4-19db7059dc37\") " pod="openshift-ingress/payload-processing-58f86c7798-kk64t" Jul 07 07:08:55.615175 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.615157 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c1d716f-4fcd-4bf9-bda4-19db7059dc37-plugins-config-volume\") pod \"payload-processing-58f86c7798-kk64t\" (UID: \"5c1d716f-4fcd-4bf9-bda4-19db7059dc37\") " pod="openshift-ingress/payload-processing-58f86c7798-kk64t" Jul 07 07:08:55.615209 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.615179 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cb8b73d-18bb-4ca8-b2af-a00c971c928d-plugins-config-volume\") pod \"payload-pre-processing-7964457696-44lzc\" (UID: \"2cb8b73d-18bb-4ca8-b2af-a00c971c928d\") " pod="openshift-ingress/payload-pre-processing-7964457696-44lzc" Jul 07 07:08:55.622945 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.622913 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dfdx\" (UniqueName: \"kubernetes.io/projected/2cb8b73d-18bb-4ca8-b2af-a00c971c928d-kube-api-access-5dfdx\") pod \"payload-pre-processing-7964457696-44lzc\" (UID: \"2cb8b73d-18bb-4ca8-b2af-a00c971c928d\") " pod="openshift-ingress/payload-pre-processing-7964457696-44lzc" Jul 07 07:08:55.622945 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.622926 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltj7q\" (UniqueName: \"kubernetes.io/projected/5c1d716f-4fcd-4bf9-bda4-19db7059dc37-kube-api-access-ltj7q\") pod \"payload-processing-58f86c7798-kk64t\" (UID: \"5c1d716f-4fcd-4bf9-bda4-19db7059dc37\") " pod="openshift-ingress/payload-processing-58f86c7798-kk64t" Jul 07 07:08:55.635541 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.635510 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-c87c98569-w4sbj" event={"ID":"557b6ddd-adcf-4000-9175-a90c34dd4a37","Type":"ContainerStarted","Data":"674e13ca49d9a9cfb4a8fbb78cc8d7353c455c86a61d1ff4e27f25ca2adf4047"} Jul 07 07:08:55.725535 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.725500 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-44lzc" Jul 07 07:08:55.747298 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.747259 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-kk64t" Jul 07 07:08:55.888766 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.888679 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-44lzc"] Jul 07 07:08:55.891245 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:08:55.891211 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cb8b73d_18bb_4ca8_b2af_a00c971c928d.slice/crio-31391239d27e03e79f60c22652105a5d5d620baec47f3f07cc822fbeedaca204 WatchSource:0}: Error finding container 31391239d27e03e79f60c22652105a5d5d620baec47f3f07cc822fbeedaca204: Status 404 returned error can't find the container with id 31391239d27e03e79f60c22652105a5d5d620baec47f3f07cc822fbeedaca204 Jul 07 07:08:55.917304 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:55.917273 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-kk64t"] Jul 07 07:08:55.919359 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:08:55.919333 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c1d716f_4fcd_4bf9_bda4_19db7059dc37.slice/crio-8c78d34df90a5c244284f24d76203c01f7ece93f2708cf44513379bfcc483eb6 WatchSource:0}: Error finding container 8c78d34df90a5c244284f24d76203c01f7ece93f2708cf44513379bfcc483eb6: Status 404 returned error can't find the container with id 8c78d34df90a5c244284f24d76203c01f7ece93f2708cf44513379bfcc483eb6 Jul 07 07:08:56.643464 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:56.643422 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-44lzc" event={"ID":"2cb8b73d-18bb-4ca8-b2af-a00c971c928d","Type":"ContainerStarted","Data":"31391239d27e03e79f60c22652105a5d5d620baec47f3f07cc822fbeedaca204"} Jul 07 07:08:56.644942 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:56.644912 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-kk64t" event={"ID":"5c1d716f-4fcd-4bf9-bda4-19db7059dc37","Type":"ContainerStarted","Data":"8c78d34df90a5c244284f24d76203c01f7ece93f2708cf44513379bfcc483eb6"} Jul 07 07:08:58.654872 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:58.654836 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-kk64t" event={"ID":"5c1d716f-4fcd-4bf9-bda4-19db7059dc37","Type":"ContainerStarted","Data":"d83dc82c79a78433d4dfc6f4b30a388e16c2aabcd70328716c538849b5072c68"} Jul 07 07:08:58.655384 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:58.654942 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-kk64t" Jul 07 07:08:58.656470 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:58.656442 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-c87c98569-w4sbj" event={"ID":"557b6ddd-adcf-4000-9175-a90c34dd4a37","Type":"ContainerStarted","Data":"8254444d100467163ee7c11f78a43ac7d3172a45e2fcee2f3153200559778de0"} Jul 07 07:08:58.656616 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:58.656507 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-c87c98569-w4sbj" Jul 07 07:08:58.657966 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:58.657947 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-44lzc" event={"ID":"2cb8b73d-18bb-4ca8-b2af-a00c971c928d","Type":"ContainerStarted","Data":"c1895a85b96f9d5c364768d85364ec5758feb43629f9f8ad8fc5b374b4d7eea9"} Jul 07 07:08:58.658066 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:58.657998 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-44lzc" Jul 07 07:08:58.673404 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:58.673360 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-kk64t" podStartSLOduration=1.91165169 podStartE2EDuration="3.673346205s" totalImagesPullingTime="1.761694515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:08:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:08:58.672788647 +0000 UTC m=+594.008754965" watchObservedRunningTime="2026-07-07 07:08:58.673346205 +0000 UTC m=+594.009312519" Jul 07 07:08:58.691424 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:58.691384 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-44lzc" podStartSLOduration=1.899554834 podStartE2EDuration="3.691369649s" totalImagesPullingTime="1.791814815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:08:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:08:58.689843149 +0000 UTC m=+594.025809489" watchObservedRunningTime="2026-07-07 07:08:58.691369649 +0000 UTC m=+594.027335966" Jul 07 07:08:58.710664 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:08:58.710624 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-c87c98569-w4sbj" podStartSLOduration=2.301270247 podStartE2EDuration="4.710609732s" totalImagesPullingTime="2.409339485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:08:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:08:58.70926538 +0000 UTC m=+594.045231718" watchObservedRunningTime="2026-07-07 07:08:58.710609732 +0000 UTC m=+594.046576049" Jul 07 07:09:04.668679 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:04.668652 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-c87c98569-w4sbj" Jul 07 07:09:05.045126 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:05.045102 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:09:05.046108 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:05.046090 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:09:05.651520 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:05.651472 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:09:05.751449 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:05.751401 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:09:07.861854 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:07.861825 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:09:09.664937 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:09.664907 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-kk64t" Jul 07 07:09:09.667316 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:09.664981 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-44lzc" Jul 07 07:09:10.215039 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.215006 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb"] Jul 07 07:09:10.218296 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.218276 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.220967 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.220941 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 07:09:10.221103 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.221005 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 07 07:09:10.221816 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.221793 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sml68\"" Jul 07 07:09:10.221929 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.221872 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 07:09:10.232219 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.232199 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb"] Jul 07 07:09:10.241198 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.241174 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:09:10.341735 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.341701 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b85b4e24-079d-48d5-bf0b-44a35d609034-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.341915 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.341743 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b85b4e24-079d-48d5-bf0b-44a35d609034-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.341915 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.341764 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b85b4e24-079d-48d5-bf0b-44a35d609034-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.341915 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.341856 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b85b4e24-079d-48d5-bf0b-44a35d609034-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.342075 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.341917 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzx98\" (UniqueName: \"kubernetes.io/projected/b85b4e24-079d-48d5-bf0b-44a35d609034-kube-api-access-pzx98\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.342075 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.341960 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b85b4e24-079d-48d5-bf0b-44a35d609034-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.443182 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.443142 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b85b4e24-079d-48d5-bf0b-44a35d609034-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.443378 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.443197 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b85b4e24-079d-48d5-bf0b-44a35d609034-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.443378 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.443224 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b85b4e24-079d-48d5-bf0b-44a35d609034-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.443378 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.443361 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b85b4e24-079d-48d5-bf0b-44a35d609034-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.443552 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.443431 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzx98\" (UniqueName: \"kubernetes.io/projected/b85b4e24-079d-48d5-bf0b-44a35d609034-kube-api-access-pzx98\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.443552 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.443472 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b85b4e24-079d-48d5-bf0b-44a35d609034-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.443664 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.443554 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b85b4e24-079d-48d5-bf0b-44a35d609034-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.443724 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.443693 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b85b4e24-079d-48d5-bf0b-44a35d609034-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.443778 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.443758 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b85b4e24-079d-48d5-bf0b-44a35d609034-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.445370 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.445350 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b85b4e24-079d-48d5-bf0b-44a35d609034-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.445678 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.445663 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b85b4e24-079d-48d5-bf0b-44a35d609034-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.452068 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.452048 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzx98\" (UniqueName: \"kubernetes.io/projected/b85b4e24-079d-48d5-bf0b-44a35d609034-kube-api-access-pzx98\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb\" (UID: \"b85b4e24-079d-48d5-bf0b-44a35d609034\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.529154 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.529119 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:10.662454 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.662424 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb"] Jul 07 07:09:10.664539 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:09:10.664508 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb85b4e24_079d_48d5_bf0b_44a35d609034.slice/crio-ddef2b6684fbe7dd5bf90ad17f0be0691441280ec98373b63fdb4befa5d5b9ed WatchSource:0}: Error finding container ddef2b6684fbe7dd5bf90ad17f0be0691441280ec98373b63fdb4befa5d5b9ed: Status 404 returned error can't find the container with id ddef2b6684fbe7dd5bf90ad17f0be0691441280ec98373b63fdb4befa5d5b9ed Jul 07 07:09:10.718385 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:10.718351 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" event={"ID":"b85b4e24-079d-48d5-bf0b-44a35d609034","Type":"ContainerStarted","Data":"ddef2b6684fbe7dd5bf90ad17f0be0691441280ec98373b63fdb4befa5d5b9ed"} Jul 07 07:09:12.730036 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:12.730005 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" event={"ID":"b85b4e24-079d-48d5-bf0b-44a35d609034","Type":"ContainerStarted","Data":"28ad0c53191058a203c0ebdbc202603ceae126ed0e05ec30e4ce895e1a2d9df8"} Jul 07 07:09:12.752239 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:12.752172 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" podStartSLOduration=1.595863461 podStartE2EDuration="2.752155159s" totalImagesPullingTime="1.156291698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:09:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:09:12.751753655 +0000 UTC m=+608.087719986" watchObservedRunningTime="2026-07-07 07:09:12.752155159 +0000 UTC m=+608.088121478" Jul 07 07:09:18.332345 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:18.332288 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:09:20.529954 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:20.529914 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:20.530406 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:20.530004 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:20.542535 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:20.542512 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:20.775482 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:20.775451 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb" Jul 07 07:09:22.937543 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:22.937505 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:09:43.727012 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.726921 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4"] Jul 07 07:09:43.733687 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.733661 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.736043 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.736013 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 07 07:09:43.740528 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.740495 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4"] Jul 07 07:09:43.812235 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.812201 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/848a836a-7ed4-4e10-9430-66c0fe1a645d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.812446 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.812241 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpd5m\" (UniqueName: \"kubernetes.io/projected/848a836a-7ed4-4e10-9430-66c0fe1a645d-kube-api-access-tpd5m\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.812446 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.812279 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/848a836a-7ed4-4e10-9430-66c0fe1a645d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.812446 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.812311 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/848a836a-7ed4-4e10-9430-66c0fe1a645d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.812446 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.812399 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/848a836a-7ed4-4e10-9430-66c0fe1a645d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.812649 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.812486 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/848a836a-7ed4-4e10-9430-66c0fe1a645d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.815255 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.815230 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:09:43.913187 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.913150 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/848a836a-7ed4-4e10-9430-66c0fe1a645d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.913401 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.913198 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpd5m\" (UniqueName: \"kubernetes.io/projected/848a836a-7ed4-4e10-9430-66c0fe1a645d-kube-api-access-tpd5m\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.913401 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.913252 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/848a836a-7ed4-4e10-9430-66c0fe1a645d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.913401 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.913282 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/848a836a-7ed4-4e10-9430-66c0fe1a645d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.913401 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.913313 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/848a836a-7ed4-4e10-9430-66c0fe1a645d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.913401 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.913392 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/848a836a-7ed4-4e10-9430-66c0fe1a645d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.913681 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.913614 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/848a836a-7ed4-4e10-9430-66c0fe1a645d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.913681 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.913666 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/848a836a-7ed4-4e10-9430-66c0fe1a645d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.913831 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.913812 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/848a836a-7ed4-4e10-9430-66c0fe1a645d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.915584 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.915562 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/848a836a-7ed4-4e10-9430-66c0fe1a645d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.915867 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.915849 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/848a836a-7ed4-4e10-9430-66c0fe1a645d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:43.922808 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:43.922788 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpd5m\" (UniqueName: \"kubernetes.io/projected/848a836a-7ed4-4e10-9430-66c0fe1a645d-kube-api-access-tpd5m\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4\" (UID: \"848a836a-7ed4-4e10-9430-66c0fe1a645d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:44.044675 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:44.044641 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:44.181895 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:44.181840 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4"] Jul 07 07:09:44.184306 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:09:44.184277 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod848a836a_7ed4_4e10_9430_66c0fe1a645d.slice/crio-7b744cec076dbb4880ea83cca0a0ce25389742ab4e3b0b68a93995d68943c219 WatchSource:0}: Error finding container 7b744cec076dbb4880ea83cca0a0ce25389742ab4e3b0b68a93995d68943c219: Status 404 returned error can't find the container with id 7b744cec076dbb4880ea83cca0a0ce25389742ab4e3b0b68a93995d68943c219 Jul 07 07:09:44.185691 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:44.185670 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 07:09:44.877242 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:44.877210 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" event={"ID":"848a836a-7ed4-4e10-9430-66c0fe1a645d","Type":"ContainerStarted","Data":"cf52b829d95898185f06dc5db80387dcc9a571d58371de2e516d74325a32918c"} Jul 07 07:09:44.877647 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:44.877257 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" event={"ID":"848a836a-7ed4-4e10-9430-66c0fe1a645d","Type":"ContainerStarted","Data":"7b744cec076dbb4880ea83cca0a0ce25389742ab4e3b0b68a93995d68943c219"} Jul 07 07:09:44.909344 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:44.909280 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" podStartSLOduration=1.7126542439999999 podStartE2EDuration="1.909265039s" totalImagesPullingTime="196.610795ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:09:44.908697839 +0000 UTC m=+640.244664158" watchObservedRunningTime="2026-07-07 07:09:44.909265039 +0000 UTC m=+640.245231356" Jul 07 07:09:47.122708 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:47.122678 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:09:54.045552 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:54.045502 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:54.046168 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:54.045621 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:54.057784 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:54.057754 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:54.934631 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:54.934599 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4" Jul 07 07:09:57.097759 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.097725 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w"] Jul 07 07:09:57.102172 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.102153 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.104545 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.104522 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 07 07:09:57.112934 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.112908 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w"] Jul 07 07:09:57.228235 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.228201 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr4dg\" (UniqueName: \"kubernetes.io/projected/a11a1121-6fe9-4579-87f2-344f019c2585-kube-api-access-qr4dg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.228462 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.228258 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a11a1121-6fe9-4579-87f2-344f019c2585-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.228462 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.228281 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a11a1121-6fe9-4579-87f2-344f019c2585-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.228462 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.228315 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a11a1121-6fe9-4579-87f2-344f019c2585-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.228462 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.228410 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a11a1121-6fe9-4579-87f2-344f019c2585-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.228651 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.228519 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a11a1121-6fe9-4579-87f2-344f019c2585-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.228726 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.228708 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:09:57.329266 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.329235 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr4dg\" (UniqueName: \"kubernetes.io/projected/a11a1121-6fe9-4579-87f2-344f019c2585-kube-api-access-qr4dg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.329479 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.329275 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a11a1121-6fe9-4579-87f2-344f019c2585-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.329479 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.329304 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a11a1121-6fe9-4579-87f2-344f019c2585-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.329479 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.329367 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a11a1121-6fe9-4579-87f2-344f019c2585-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.329479 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.329407 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a11a1121-6fe9-4579-87f2-344f019c2585-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.329479 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.329470 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a11a1121-6fe9-4579-87f2-344f019c2585-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.329752 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.329663 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a11a1121-6fe9-4579-87f2-344f019c2585-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.329752 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.329733 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a11a1121-6fe9-4579-87f2-344f019c2585-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.329856 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.329825 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a11a1121-6fe9-4579-87f2-344f019c2585-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.331623 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.331602 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a11a1121-6fe9-4579-87f2-344f019c2585-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.331894 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.331879 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a11a1121-6fe9-4579-87f2-344f019c2585-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.351788 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.351727 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr4dg\" (UniqueName: \"kubernetes.io/projected/a11a1121-6fe9-4579-87f2-344f019c2585-kube-api-access-qr4dg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9pj6w\" (UID: \"a11a1121-6fe9-4579-87f2-344f019c2585\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.415711 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.415679 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:09:57.563914 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.563893 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w"] Jul 07 07:09:57.565572 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:09:57.565544 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda11a1121_6fe9_4579_87f2_344f019c2585.slice/crio-94763b6480d3ba5497f460d32bbe46e08bfa9fae26ac3029ed639441a2b1a888 WatchSource:0}: Error finding container 94763b6480d3ba5497f460d32bbe46e08bfa9fae26ac3029ed639441a2b1a888: Status 404 returned error can't find the container with id 94763b6480d3ba5497f460d32bbe46e08bfa9fae26ac3029ed639441a2b1a888 Jul 07 07:09:57.943669 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.943630 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" event={"ID":"a11a1121-6fe9-4579-87f2-344f019c2585","Type":"ContainerStarted","Data":"aa0028357f2bd4650b024102272b868e92bc45875265eead285ac498bcbaa3a9"} Jul 07 07:09:57.943851 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.943721 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" event={"ID":"a11a1121-6fe9-4579-87f2-344f019c2585","Type":"ContainerStarted","Data":"94763b6480d3ba5497f460d32bbe46e08bfa9fae26ac3029ed639441a2b1a888"} Jul 07 07:09:57.970278 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:09:57.970223 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" podStartSLOduration=0.776286614 podStartE2EDuration="970.2094ms" totalImagesPullingTime="193.922786ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:09:57.968018474 +0000 UTC m=+653.303984816" watchObservedRunningTime="2026-07-07 07:09:57.9702094 +0000 UTC m=+653.306175716" Jul 07 07:10:07.415904 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:10:07.415864 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:10:07.416313 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:10:07.415931 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:10:07.428381 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:10:07.428353 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:10:08.000748 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:10:08.000720 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9pj6w" Jul 07 07:10:08.165052 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:10:08.165016 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:10:18.111198 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:10:18.111167 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:10:51.575123 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:10:51.575082 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:10:51.679492 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:10:51.679456 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:10:52.760812 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:10:52.760781 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:10:52.881309 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:10:52.881277 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:10:59.279866 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:10:59.279789 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:11:01.274834 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:11:01.274799 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:14.834740 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:14.834711 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:14.913142 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:14.913114 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c8d7dbb94-r4mw8"] Jul 07 07:12:14.913359 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:14.913314 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" podUID="6c6f195f-595c-4ce1-8a0e-6f1aed0f1522" containerName="manager" containerID="cri-o://90d010498971500bde994b07dee0e84d69994efa036f4de2ff256c7a38185ae7" gracePeriod=30 Jul 07 07:12:15.162039 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:15.162017 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" Jul 07 07:12:15.226376 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:15.226338 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c6f195f-595c-4ce1-8a0e-6f1aed0f1522-webhook-cert\") pod \"6c6f195f-595c-4ce1-8a0e-6f1aed0f1522\" (UID: \"6c6f195f-595c-4ce1-8a0e-6f1aed0f1522\") " Jul 07 07:12:15.226519 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:15.226399 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9q2w\" (UniqueName: \"kubernetes.io/projected/6c6f195f-595c-4ce1-8a0e-6f1aed0f1522-kube-api-access-h9q2w\") pod \"6c6f195f-595c-4ce1-8a0e-6f1aed0f1522\" (UID: \"6c6f195f-595c-4ce1-8a0e-6f1aed0f1522\") " Jul 07 07:12:15.228369 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:15.228315 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c6f195f-595c-4ce1-8a0e-6f1aed0f1522-kube-api-access-h9q2w" (OuterVolumeSpecName: "kube-api-access-h9q2w") pod "6c6f195f-595c-4ce1-8a0e-6f1aed0f1522" (UID: "6c6f195f-595c-4ce1-8a0e-6f1aed0f1522"). InnerVolumeSpecName "kube-api-access-h9q2w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:12:15.228472 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:15.228414 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c6f195f-595c-4ce1-8a0e-6f1aed0f1522-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6c6f195f-595c-4ce1-8a0e-6f1aed0f1522" (UID: "6c6f195f-595c-4ce1-8a0e-6f1aed0f1522"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:12:15.327277 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:15.327245 2573 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c6f195f-595c-4ce1-8a0e-6f1aed0f1522-webhook-cert\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:12:15.327277 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:15.327272 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9q2w\" (UniqueName: \"kubernetes.io/projected/6c6f195f-595c-4ce1-8a0e-6f1aed0f1522-kube-api-access-h9q2w\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:12:15.558585 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:15.558557 2573 generic.go:356] "Generic (PLEG): container finished" podID="6c6f195f-595c-4ce1-8a0e-6f1aed0f1522" containerID="90d010498971500bde994b07dee0e84d69994efa036f4de2ff256c7a38185ae7" exitCode=0 Jul 07 07:12:15.558746 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:15.558599 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" Jul 07 07:12:15.558746 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:15.558609 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" event={"ID":"6c6f195f-595c-4ce1-8a0e-6f1aed0f1522","Type":"ContainerDied","Data":"90d010498971500bde994b07dee0e84d69994efa036f4de2ff256c7a38185ae7"} Jul 07 07:12:15.558746 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:15.558649 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8d7dbb94-r4mw8" event={"ID":"6c6f195f-595c-4ce1-8a0e-6f1aed0f1522","Type":"ContainerDied","Data":"8c9d93a78929d57cf29004965a3b9c073409235a0d6c2f963346de7b89b692eb"} Jul 07 07:12:15.558746 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:15.558671 2573 scope.go:117] "RemoveContainer" containerID="90d010498971500bde994b07dee0e84d69994efa036f4de2ff256c7a38185ae7" Jul 07 07:12:15.567382 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:15.567361 2573 scope.go:117] "RemoveContainer" containerID="90d010498971500bde994b07dee0e84d69994efa036f4de2ff256c7a38185ae7" Jul 07 07:12:15.567597 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:12:15.567581 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90d010498971500bde994b07dee0e84d69994efa036f4de2ff256c7a38185ae7\": container with ID starting with 90d010498971500bde994b07dee0e84d69994efa036f4de2ff256c7a38185ae7 not found: ID does not exist" containerID="90d010498971500bde994b07dee0e84d69994efa036f4de2ff256c7a38185ae7" Jul 07 07:12:15.567642 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:15.567603 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90d010498971500bde994b07dee0e84d69994efa036f4de2ff256c7a38185ae7"} err="failed to get container status \"90d010498971500bde994b07dee0e84d69994efa036f4de2ff256c7a38185ae7\": rpc error: code = NotFound desc = could not find container \"90d010498971500bde994b07dee0e84d69994efa036f4de2ff256c7a38185ae7\": container with ID starting with 90d010498971500bde994b07dee0e84d69994efa036f4de2ff256c7a38185ae7 not found: ID does not exist" Jul 07 07:12:15.582729 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:15.582704 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c8d7dbb94-r4mw8"] Jul 07 07:12:15.589245 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:15.589223 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c8d7dbb94-r4mw8"] Jul 07 07:12:16.413550 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.413517 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c8d7dbb94-2jrqx"] Jul 07 07:12:16.413917 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.413863 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c6f195f-595c-4ce1-8a0e-6f1aed0f1522" containerName="manager" Jul 07 07:12:16.413917 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.413874 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c6f195f-595c-4ce1-8a0e-6f1aed0f1522" containerName="manager" Jul 07 07:12:16.413985 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.413955 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c6f195f-595c-4ce1-8a0e-6f1aed0f1522" containerName="manager" Jul 07 07:12:16.417914 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.417897 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8d7dbb94-2jrqx" Jul 07 07:12:16.421306 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.421284 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 07:12:16.421430 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.421284 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 07:12:16.422232 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.422217 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rdpdd\"" Jul 07 07:12:16.433737 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.433670 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c8d7dbb94-2jrqx"] Jul 07 07:12:16.537592 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.537559 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/906aa68f-c3b7-498b-8dae-4298deca39bd-webhook-cert\") pod \"maas-controller-6c8d7dbb94-2jrqx\" (UID: \"906aa68f-c3b7-498b-8dae-4298deca39bd\") " pod="opendatahub/maas-controller-6c8d7dbb94-2jrqx" Jul 07 07:12:16.537723 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.537598 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-822c4\" (UniqueName: \"kubernetes.io/projected/906aa68f-c3b7-498b-8dae-4298deca39bd-kube-api-access-822c4\") pod \"maas-controller-6c8d7dbb94-2jrqx\" (UID: \"906aa68f-c3b7-498b-8dae-4298deca39bd\") " pod="opendatahub/maas-controller-6c8d7dbb94-2jrqx" Jul 07 07:12:16.597110 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.597080 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:16.638582 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.638558 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/906aa68f-c3b7-498b-8dae-4298deca39bd-webhook-cert\") pod \"maas-controller-6c8d7dbb94-2jrqx\" (UID: \"906aa68f-c3b7-498b-8dae-4298deca39bd\") " pod="opendatahub/maas-controller-6c8d7dbb94-2jrqx" Jul 07 07:12:16.638703 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.638592 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-822c4\" (UniqueName: \"kubernetes.io/projected/906aa68f-c3b7-498b-8dae-4298deca39bd-kube-api-access-822c4\") pod \"maas-controller-6c8d7dbb94-2jrqx\" (UID: \"906aa68f-c3b7-498b-8dae-4298deca39bd\") " pod="opendatahub/maas-controller-6c8d7dbb94-2jrqx" Jul 07 07:12:16.640927 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.640901 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/906aa68f-c3b7-498b-8dae-4298deca39bd-webhook-cert\") pod \"maas-controller-6c8d7dbb94-2jrqx\" (UID: \"906aa68f-c3b7-498b-8dae-4298deca39bd\") " pod="opendatahub/maas-controller-6c8d7dbb94-2jrqx" Jul 07 07:12:16.650346 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.650300 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-822c4\" (UniqueName: \"kubernetes.io/projected/906aa68f-c3b7-498b-8dae-4298deca39bd-kube-api-access-822c4\") pod \"maas-controller-6c8d7dbb94-2jrqx\" (UID: \"906aa68f-c3b7-498b-8dae-4298deca39bd\") " pod="opendatahub/maas-controller-6c8d7dbb94-2jrqx" Jul 07 07:12:16.727446 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.727384 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8d7dbb94-2jrqx" Jul 07 07:12:16.856625 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:16.856603 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c8d7dbb94-2jrqx"] Jul 07 07:12:16.858817 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:12:16.858789 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod906aa68f_c3b7_498b_8dae_4298deca39bd.slice/crio-e93860df8e6720a4f0934d9da8361ec674b392c56acf640d07b87f54a3569b06 WatchSource:0}: Error finding container e93860df8e6720a4f0934d9da8361ec674b392c56acf640d07b87f54a3569b06: Status 404 returned error can't find the container with id e93860df8e6720a4f0934d9da8361ec674b392c56acf640d07b87f54a3569b06 Jul 07 07:12:17.107745 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:17.107710 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c6f195f-595c-4ce1-8a0e-6f1aed0f1522" path="/var/lib/kubelet/pods/6c6f195f-595c-4ce1-8a0e-6f1aed0f1522/volumes" Jul 07 07:12:17.569599 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:17.569564 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8d7dbb94-2jrqx" event={"ID":"906aa68f-c3b7-498b-8dae-4298deca39bd","Type":"ContainerStarted","Data":"e93860df8e6720a4f0934d9da8361ec674b392c56acf640d07b87f54a3569b06"} Jul 07 07:12:18.575424 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:18.575392 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8d7dbb94-2jrqx" event={"ID":"906aa68f-c3b7-498b-8dae-4298deca39bd","Type":"ContainerStarted","Data":"127e4e6000aedeadcf6d2f410f366dde0b2fc31f09b6a92958c50ed7b30c233a"} Jul 07 07:12:18.575822 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:18.575490 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c8d7dbb94-2jrqx" Jul 07 07:12:18.597236 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:18.597188 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c8d7dbb94-2jrqx" podStartSLOduration=1.6561331099999999 podStartE2EDuration="2.597174195s" totalImagesPullingTime="941.041085ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:12:18.595009828 +0000 UTC m=+793.930976144" watchObservedRunningTime="2026-07-07 07:12:18.597174195 +0000 UTC m=+793.933140512" Jul 07 07:12:29.588245 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:29.588172 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c8d7dbb94-2jrqx" Jul 07 07:12:57.862023 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:57.861990 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:57.956978 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:57.956944 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:58.094347 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:12:58.094299 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:13:04.544359 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:13:04.544306 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:13:04.613029 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:13:04.612988 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:13:04.724346 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:13:04.724287 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:13:04.923897 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:13:04.923808 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:13:10.862845 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:13:10.862810 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:13:10.954868 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:13:10.954840 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:14:05.071937 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:14:05.071896 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:14:05.072934 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:14:05.072911 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:14:54.181488 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:14:54.181451 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:14:56.198576 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:14:56.198522 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:16:41.472517 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:16:41.472472 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:16:41.539494 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:16:41.539464 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:17:01.390546 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:17:01.390449 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:18:46.129619 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:18:46.129533 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:19:05.097201 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:19:05.097167 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:19:05.099225 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:19:05.099202 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:23:51.210216 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:51.210175 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:23:52.708508 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:52.708475 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:23:58.660929 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:58.660862 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:23:58.763755 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:58.763722 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:23:59.190015 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.189982 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:23:59.285887 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.285857 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:23:59.402374 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.402336 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2"] Jul 07 07:23:59.406222 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.406202 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" Jul 07 07:23:59.408916 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.408896 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-646cbfff-serving-cert\"" Jul 07 07:23:59.416453 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.416427 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2"] Jul 07 07:23:59.498016 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.497976 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:23:59.552103 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.552071 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-maas-api-tls\") pod \"maas-api-e2e-ait-646cbfff-66b4f65875-779m2\" (UID: \"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b\") " pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" Jul 07 07:23:59.552241 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.552128 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-openshift-service-ca\") pod \"maas-api-e2e-ait-646cbfff-66b4f65875-779m2\" (UID: \"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b\") " pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" Jul 07 07:23:59.552241 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.552233 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99g6z\" (UniqueName: \"kubernetes.io/projected/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-kube-api-access-99g6z\") pod \"maas-api-e2e-ait-646cbfff-66b4f65875-779m2\" (UID: \"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b\") " pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" Jul 07 07:23:59.617483 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.617457 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:23:59.652990 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.652961 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99g6z\" (UniqueName: \"kubernetes.io/projected/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-kube-api-access-99g6z\") pod \"maas-api-e2e-ait-646cbfff-66b4f65875-779m2\" (UID: \"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b\") " pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" Jul 07 07:23:59.653151 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.652998 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-maas-api-tls\") pod \"maas-api-e2e-ait-646cbfff-66b4f65875-779m2\" (UID: \"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b\") " pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" Jul 07 07:23:59.653151 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.653028 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-openshift-service-ca\") pod \"maas-api-e2e-ait-646cbfff-66b4f65875-779m2\" (UID: \"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b\") " pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" Jul 07 07:23:59.653877 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.653851 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-openshift-service-ca\") pod \"maas-api-e2e-ait-646cbfff-66b4f65875-779m2\" (UID: \"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b\") " pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" Jul 07 07:23:59.655627 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.655606 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-maas-api-tls\") pod \"maas-api-e2e-ait-646cbfff-66b4f65875-779m2\" (UID: \"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b\") " pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" Jul 07 07:23:59.662703 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.662678 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99g6z\" (UniqueName: \"kubernetes.io/projected/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-kube-api-access-99g6z\") pod \"maas-api-e2e-ait-646cbfff-66b4f65875-779m2\" (UID: \"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b\") " pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" Jul 07 07:23:59.718616 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.718589 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" Jul 07 07:23:59.857827 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.857795 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2"] Jul 07 07:23:59.860177 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:23:59.860152 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eaeb4e1_956a_49f9_aeb2_bdd3e670d54b.slice/crio-7e4561482ba802cf0537504a51a3eed878eddf351be2254dca814f174f0f0aa9 WatchSource:0}: Error finding container 7e4561482ba802cf0537504a51a3eed878eddf351be2254dca814f174f0f0aa9: Status 404 returned error can't find the container with id 7e4561482ba802cf0537504a51a3eed878eddf351be2254dca814f174f0f0aa9 Jul 07 07:23:59.862053 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.862024 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 07:23:59.978649 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:23:59.978593 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:00.079169 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:00.079087 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:00.774666 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:00.774632 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" event={"ID":"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b","Type":"ContainerStarted","Data":"9ceaaf9b7fb536787cc2e37433bac031ff56917ddc26a568f431e54bfc09c834"} Jul 07 07:24:00.775130 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:00.774710 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" event={"ID":"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b","Type":"ContainerStarted","Data":"7e4561482ba802cf0537504a51a3eed878eddf351be2254dca814f174f0f0aa9"} Jul 07 07:24:00.775130 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:00.774751 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" Jul 07 07:24:00.793964 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:00.793920 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" podStartSLOduration=1.259675171 podStartE2EDuration="1.793905821s" totalImagesPullingTime="534.23065ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:23:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:24:00.793071764 +0000 UTC m=+1496.129038092" watchObservedRunningTime="2026-07-07 07:24:00.793905821 +0000 UTC m=+1496.129872137" Jul 07 07:24:01.524473 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:01.524429 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:04.299524 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:04.299483 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:04.507488 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:04.507447 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:04.604200 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:04.604126 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:04.642387 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:04.642355 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2"] Jul 07 07:24:04.642623 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:04.642554 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" podUID="4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b" containerName="maas-api" containerID="cri-o://9ceaaf9b7fb536787cc2e37433bac031ff56917ddc26a568f431e54bfc09c834" gracePeriod=30 Jul 07 07:24:04.678217 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:04.678192 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:04.795601 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:04.795570 2573 generic.go:356] "Generic (PLEG): container finished" podID="4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b" containerID="9ceaaf9b7fb536787cc2e37433bac031ff56917ddc26a568f431e54bfc09c834" exitCode=0 Jul 07 07:24:04.795766 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:04.795640 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" event={"ID":"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b","Type":"ContainerDied","Data":"9ceaaf9b7fb536787cc2e37433bac031ff56917ddc26a568f431e54bfc09c834"} Jul 07 07:24:04.890903 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:04.890880 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" Jul 07 07:24:05.001254 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:05.001214 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-openshift-service-ca\") pod \"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b\" (UID: \"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b\") " Jul 07 07:24:05.001476 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:05.001274 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-maas-api-tls\") pod \"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b\" (UID: \"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b\") " Jul 07 07:24:05.001476 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:05.001309 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99g6z\" (UniqueName: \"kubernetes.io/projected/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-kube-api-access-99g6z\") pod \"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b\" (UID: \"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b\") " Jul 07 07:24:05.001608 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:05.001583 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b" (UID: "4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:24:05.003374 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:05.003354 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b" (UID: "4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:24:05.003429 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:05.003356 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-kube-api-access-99g6z" (OuterVolumeSpecName: "kube-api-access-99g6z") pod "4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b" (UID: "4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b"). InnerVolumeSpecName "kube-api-access-99g6z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:24:05.102736 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:05.102703 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-openshift-service-ca\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:24:05.102736 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:05.102729 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-maas-api-tls\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:24:05.102736 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:05.102738 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99g6z\" (UniqueName: \"kubernetes.io/projected/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b-kube-api-access-99g6z\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:24:05.131091 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:05.131060 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:24:05.134552 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:05.134531 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:24:05.801963 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:05.801935 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" Jul 07 07:24:05.802447 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:05.801931 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2" event={"ID":"4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b","Type":"ContainerDied","Data":"7e4561482ba802cf0537504a51a3eed878eddf351be2254dca814f174f0f0aa9"} Jul 07 07:24:05.802447 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:05.802066 2573 scope.go:117] "RemoveContainer" containerID="9ceaaf9b7fb536787cc2e37433bac031ff56917ddc26a568f431e54bfc09c834" Jul 07 07:24:05.821895 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:05.821873 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2"] Jul 07 07:24:05.826663 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:05.826642 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-646cbfff-66b4f65875-779m2"] Jul 07 07:24:07.107744 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:07.107711 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b" path="/var/lib/kubelet/pods/4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b/volumes" Jul 07 07:24:07.486163 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:07.486125 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:10.758382 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:10.758352 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:10.859036 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:10.859001 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:10.994556 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:10.994526 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:11.069189 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:11.069110 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:12.171348 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.171277 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:12.273133 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.273102 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:12.304913 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.304885 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb"] Jul 07 07:24:12.305256 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.305242 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b" containerName="maas-api" Jul 07 07:24:12.305256 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.305257 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b" containerName="maas-api" Jul 07 07:24:12.305367 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.305342 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eaeb4e1-956a-49f9-aeb2-bdd3e670d54b" containerName="maas-api" Jul 07 07:24:12.307823 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.307807 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" Jul 07 07:24:12.310668 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.310649 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-2cb4b8b0-serving-cert\"" Jul 07 07:24:12.321125 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.321104 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb"] Jul 07 07:24:12.409956 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.409907 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:12.467648 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.467568 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a24558-6940-405d-8314-6fbbca6d0578-maas-api-tls\") pod \"maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb\" (UID: \"98a24558-6940-405d-8314-6fbbca6d0578\") " pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" Jul 07 07:24:12.467802 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.467688 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98a24558-6940-405d-8314-6fbbca6d0578-openshift-service-ca\") pod \"maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb\" (UID: \"98a24558-6940-405d-8314-6fbbca6d0578\") " pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" Jul 07 07:24:12.467802 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.467725 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hwrn\" (UniqueName: \"kubernetes.io/projected/98a24558-6940-405d-8314-6fbbca6d0578-kube-api-access-4hwrn\") pod \"maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb\" (UID: \"98a24558-6940-405d-8314-6fbbca6d0578\") " pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" Jul 07 07:24:12.568281 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.568243 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98a24558-6940-405d-8314-6fbbca6d0578-openshift-service-ca\") pod \"maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb\" (UID: \"98a24558-6940-405d-8314-6fbbca6d0578\") " pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" Jul 07 07:24:12.568482 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.568289 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hwrn\" (UniqueName: \"kubernetes.io/projected/98a24558-6940-405d-8314-6fbbca6d0578-kube-api-access-4hwrn\") pod \"maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb\" (UID: \"98a24558-6940-405d-8314-6fbbca6d0578\") " pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" Jul 07 07:24:12.568482 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.568365 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a24558-6940-405d-8314-6fbbca6d0578-maas-api-tls\") pod \"maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb\" (UID: \"98a24558-6940-405d-8314-6fbbca6d0578\") " pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" Jul 07 07:24:12.568600 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:24:12.568516 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-2cb4b8b0-serving-cert: secret "maas-api-e2e-ait-2cb4b8b0-serving-cert" not found Jul 07 07:24:12.568600 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:24:12.568593 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98a24558-6940-405d-8314-6fbbca6d0578-maas-api-tls podName:98a24558-6940-405d-8314-6fbbca6d0578 nodeName:}" failed. No retries permitted until 2026-07-07 07:24:13.068571132 +0000 UTC m=+1508.404537445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/98a24558-6940-405d-8314-6fbbca6d0578-maas-api-tls") pod "maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" (UID: "98a24558-6940-405d-8314-6fbbca6d0578") : secret "maas-api-e2e-ait-2cb4b8b0-serving-cert" not found Jul 07 07:24:12.568887 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.568866 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98a24558-6940-405d-8314-6fbbca6d0578-openshift-service-ca\") pod \"maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb\" (UID: \"98a24558-6940-405d-8314-6fbbca6d0578\") " pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" Jul 07 07:24:12.582785 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.582763 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hwrn\" (UniqueName: \"kubernetes.io/projected/98a24558-6940-405d-8314-6fbbca6d0578-kube-api-access-4hwrn\") pod \"maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb\" (UID: \"98a24558-6940-405d-8314-6fbbca6d0578\") " pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" Jul 07 07:24:12.942926 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:12.942877 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:13.044139 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:13.044099 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:13.072756 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:13.072719 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a24558-6940-405d-8314-6fbbca6d0578-maas-api-tls\") pod \"maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb\" (UID: \"98a24558-6940-405d-8314-6fbbca6d0578\") " pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" Jul 07 07:24:13.075101 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:13.075077 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a24558-6940-405d-8314-6fbbca6d0578-maas-api-tls\") pod \"maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb\" (UID: \"98a24558-6940-405d-8314-6fbbca6d0578\") " pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" Jul 07 07:24:13.143929 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:13.143897 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:13.217399 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:13.217307 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" Jul 07 07:24:13.359884 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:13.359863 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb"] Jul 07 07:24:13.362350 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:24:13.362301 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a24558_6940_405d_8314_6fbbca6d0578.slice/crio-04cda4b4ad88184239c1feecf0bcb9baf7261ec245caf0b2e65e4fbc0f079c62 WatchSource:0}: Error finding container 04cda4b4ad88184239c1feecf0bcb9baf7261ec245caf0b2e65e4fbc0f079c62: Status 404 returned error can't find the container with id 04cda4b4ad88184239c1feecf0bcb9baf7261ec245caf0b2e65e4fbc0f079c62 Jul 07 07:24:13.840392 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:13.840355 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" event={"ID":"98a24558-6940-405d-8314-6fbbca6d0578","Type":"ContainerStarted","Data":"9e20d821e436c0c4eee5925de191e3242d20b987d49620f35295db3e5e625b31"} Jul 07 07:24:13.840569 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:13.840428 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" Jul 07 07:24:13.840569 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:13.840452 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" event={"ID":"98a24558-6940-405d-8314-6fbbca6d0578","Type":"ContainerStarted","Data":"04cda4b4ad88184239c1feecf0bcb9baf7261ec245caf0b2e65e4fbc0f079c62"} Jul 07 07:24:13.861948 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:13.861902 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" podStartSLOduration=1.485651372 podStartE2EDuration="1.861886868s" totalImagesPullingTime="376.235496ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:24:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:24:13.860146251 +0000 UTC m=+1509.196112619" watchObservedRunningTime="2026-07-07 07:24:13.861886868 +0000 UTC m=+1509.197853186" Jul 07 07:24:14.354179 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:14.354151 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:17.209976 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.209939 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:17.553561 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.553521 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb"] Jul 07 07:24:17.553803 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.553769 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" podUID="98a24558-6940-405d-8314-6fbbca6d0578" containerName="maas-api" containerID="cri-o://9e20d821e436c0c4eee5925de191e3242d20b987d49620f35295db3e5e625b31" gracePeriod=30 Jul 07 07:24:17.577211 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.577182 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:17.803644 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.803582 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" Jul 07 07:24:17.859520 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.859490 2573 generic.go:356] "Generic (PLEG): container finished" podID="98a24558-6940-405d-8314-6fbbca6d0578" containerID="9e20d821e436c0c4eee5925de191e3242d20b987d49620f35295db3e5e625b31" exitCode=0 Jul 07 07:24:17.859641 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.859542 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" Jul 07 07:24:17.859641 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.859571 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" event={"ID":"98a24558-6940-405d-8314-6fbbca6d0578","Type":"ContainerDied","Data":"9e20d821e436c0c4eee5925de191e3242d20b987d49620f35295db3e5e625b31"} Jul 07 07:24:17.859641 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.859619 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb" event={"ID":"98a24558-6940-405d-8314-6fbbca6d0578","Type":"ContainerDied","Data":"04cda4b4ad88184239c1feecf0bcb9baf7261ec245caf0b2e65e4fbc0f079c62"} Jul 07 07:24:17.859641 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.859642 2573 scope.go:117] "RemoveContainer" containerID="9e20d821e436c0c4eee5925de191e3242d20b987d49620f35295db3e5e625b31" Jul 07 07:24:17.868142 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.868121 2573 scope.go:117] "RemoveContainer" containerID="9e20d821e436c0c4eee5925de191e3242d20b987d49620f35295db3e5e625b31" Jul 07 07:24:17.868415 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:24:17.868390 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e20d821e436c0c4eee5925de191e3242d20b987d49620f35295db3e5e625b31\": container with ID starting with 9e20d821e436c0c4eee5925de191e3242d20b987d49620f35295db3e5e625b31 not found: ID does not exist" containerID="9e20d821e436c0c4eee5925de191e3242d20b987d49620f35295db3e5e625b31" Jul 07 07:24:17.868475 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.868425 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e20d821e436c0c4eee5925de191e3242d20b987d49620f35295db3e5e625b31"} err="failed to get container status \"9e20d821e436c0c4eee5925de191e3242d20b987d49620f35295db3e5e625b31\": rpc error: code = NotFound desc = could not find container \"9e20d821e436c0c4eee5925de191e3242d20b987d49620f35295db3e5e625b31\": container with ID starting with 9e20d821e436c0c4eee5925de191e3242d20b987d49620f35295db3e5e625b31 not found: ID does not exist" Jul 07 07:24:17.912883 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.912861 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a24558-6940-405d-8314-6fbbca6d0578-maas-api-tls\") pod \"98a24558-6940-405d-8314-6fbbca6d0578\" (UID: \"98a24558-6940-405d-8314-6fbbca6d0578\") " Jul 07 07:24:17.912974 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.912891 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98a24558-6940-405d-8314-6fbbca6d0578-openshift-service-ca\") pod \"98a24558-6940-405d-8314-6fbbca6d0578\" (UID: \"98a24558-6940-405d-8314-6fbbca6d0578\") " Jul 07 07:24:17.912974 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.912918 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hwrn\" (UniqueName: \"kubernetes.io/projected/98a24558-6940-405d-8314-6fbbca6d0578-kube-api-access-4hwrn\") pod \"98a24558-6940-405d-8314-6fbbca6d0578\" (UID: \"98a24558-6940-405d-8314-6fbbca6d0578\") " Jul 07 07:24:17.913222 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.913202 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98a24558-6940-405d-8314-6fbbca6d0578-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "98a24558-6940-405d-8314-6fbbca6d0578" (UID: "98a24558-6940-405d-8314-6fbbca6d0578"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:24:17.914901 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.914879 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98a24558-6940-405d-8314-6fbbca6d0578-kube-api-access-4hwrn" (OuterVolumeSpecName: "kube-api-access-4hwrn") pod "98a24558-6940-405d-8314-6fbbca6d0578" (UID: "98a24558-6940-405d-8314-6fbbca6d0578"). InnerVolumeSpecName "kube-api-access-4hwrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:24:17.914975 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:17.914906 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98a24558-6940-405d-8314-6fbbca6d0578-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "98a24558-6940-405d-8314-6fbbca6d0578" (UID: "98a24558-6940-405d-8314-6fbbca6d0578"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:24:18.014242 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:18.014211 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hwrn\" (UniqueName: \"kubernetes.io/projected/98a24558-6940-405d-8314-6fbbca6d0578-kube-api-access-4hwrn\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:24:18.014242 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:18.014236 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a24558-6940-405d-8314-6fbbca6d0578-maas-api-tls\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:24:18.014242 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:18.014245 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98a24558-6940-405d-8314-6fbbca6d0578-openshift-service-ca\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:24:18.185441 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:18.185412 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb"] Jul 07 07:24:18.189549 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:18.189527 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-2cb4b8b0-7879cd6bb7-f4xrb"] Jul 07 07:24:19.108083 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:19.108040 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98a24558-6940-405d-8314-6fbbca6d0578" path="/var/lib/kubelet/pods/98a24558-6940-405d-8314-6fbbca6d0578/volumes" Jul 07 07:24:23.297933 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:23.297901 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:23.405238 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:23.405204 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:25.412081 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:25.412051 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:26.922311 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:26.922233 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:27.024481 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:27.024447 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:29.943679 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:29.943644 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:30.044204 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:30.044176 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:30.147785 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:30.147752 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:30.214438 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:30.214360 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:31.266365 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.266315 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:31.368030 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.368001 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:31.433299 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.433268 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm"] Jul 07 07:24:31.433706 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.433687 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98a24558-6940-405d-8314-6fbbca6d0578" containerName="maas-api" Jul 07 07:24:31.433800 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.433707 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="98a24558-6940-405d-8314-6fbbca6d0578" containerName="maas-api" Jul 07 07:24:31.433859 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.433840 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="98a24558-6940-405d-8314-6fbbca6d0578" containerName="maas-api" Jul 07 07:24:31.436596 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.436574 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" Jul 07 07:24:31.440035 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.440014 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-3b8063ff-serving-cert\"" Jul 07 07:24:31.454810 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.454786 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm"] Jul 07 07:24:31.530126 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.530038 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55d5dc59-6c63-4519-b489-8f42891ea2be-openshift-service-ca\") pod \"maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm\" (UID: \"55d5dc59-6c63-4519-b489-8f42891ea2be\") " pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" Jul 07 07:24:31.530126 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.530081 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwnkg\" (UniqueName: \"kubernetes.io/projected/55d5dc59-6c63-4519-b489-8f42891ea2be-kube-api-access-kwnkg\") pod \"maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm\" (UID: \"55d5dc59-6c63-4519-b489-8f42891ea2be\") " pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" Jul 07 07:24:31.530346 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.530184 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55d5dc59-6c63-4519-b489-8f42891ea2be-maas-api-tls\") pod \"maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm\" (UID: \"55d5dc59-6c63-4519-b489-8f42891ea2be\") " pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" Jul 07 07:24:31.531631 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.531601 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:31.631260 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.631223 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55d5dc59-6c63-4519-b489-8f42891ea2be-maas-api-tls\") pod \"maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm\" (UID: \"55d5dc59-6c63-4519-b489-8f42891ea2be\") " pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" Jul 07 07:24:31.631477 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.631284 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55d5dc59-6c63-4519-b489-8f42891ea2be-openshift-service-ca\") pod \"maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm\" (UID: \"55d5dc59-6c63-4519-b489-8f42891ea2be\") " pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" Jul 07 07:24:31.631477 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.631402 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwnkg\" (UniqueName: \"kubernetes.io/projected/55d5dc59-6c63-4519-b489-8f42891ea2be-kube-api-access-kwnkg\") pod \"maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm\" (UID: \"55d5dc59-6c63-4519-b489-8f42891ea2be\") " pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" Jul 07 07:24:31.631477 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:24:31.631426 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-3b8063ff-serving-cert: secret "maas-api-e2e-derive-3b8063ff-serving-cert" not found Jul 07 07:24:31.631642 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:24:31.631518 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55d5dc59-6c63-4519-b489-8f42891ea2be-maas-api-tls podName:55d5dc59-6c63-4519-b489-8f42891ea2be nodeName:}" failed. No retries permitted until 2026-07-07 07:24:32.131496416 +0000 UTC m=+1527.467462711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/55d5dc59-6c63-4519-b489-8f42891ea2be-maas-api-tls") pod "maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" (UID: "55d5dc59-6c63-4519-b489-8f42891ea2be") : secret "maas-api-e2e-derive-3b8063ff-serving-cert" not found Jul 07 07:24:31.631915 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.631899 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55d5dc59-6c63-4519-b489-8f42891ea2be-openshift-service-ca\") pod \"maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm\" (UID: \"55d5dc59-6c63-4519-b489-8f42891ea2be\") " pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" Jul 07 07:24:31.646029 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.645999 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwnkg\" (UniqueName: \"kubernetes.io/projected/55d5dc59-6c63-4519-b489-8f42891ea2be-kube-api-access-kwnkg\") pod \"maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm\" (UID: \"55d5dc59-6c63-4519-b489-8f42891ea2be\") " pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" Jul 07 07:24:31.796125 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:31.796034 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:32.118631 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:32.118541 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:32.135817 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:32.135788 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55d5dc59-6c63-4519-b489-8f42891ea2be-maas-api-tls\") pod \"maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm\" (UID: \"55d5dc59-6c63-4519-b489-8f42891ea2be\") " pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" Jul 07 07:24:32.138422 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:32.138389 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55d5dc59-6c63-4519-b489-8f42891ea2be-maas-api-tls\") pod \"maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm\" (UID: \"55d5dc59-6c63-4519-b489-8f42891ea2be\") " pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" Jul 07 07:24:32.218040 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:32.218007 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:32.349537 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:32.349497 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" Jul 07 07:24:32.506895 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:32.506872 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm"] Jul 07 07:24:32.508962 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:24:32.508934 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55d5dc59_6c63_4519_b489_8f42891ea2be.slice/crio-201d1dce5a620f107031a0b5c11a10522c2eb49e43f60060264dc6bbd93a71cb WatchSource:0}: Error finding container 201d1dce5a620f107031a0b5c11a10522c2eb49e43f60060264dc6bbd93a71cb: Status 404 returned error can't find the container with id 201d1dce5a620f107031a0b5c11a10522c2eb49e43f60060264dc6bbd93a71cb Jul 07 07:24:32.940142 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:32.940110 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" event={"ID":"55d5dc59-6c63-4519-b489-8f42891ea2be","Type":"ContainerStarted","Data":"a0dba73391366f5a8292ad89df9ad9e54ea08a348a447710fc7406aa7a0cdfd3"} Jul 07 07:24:32.940309 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:32.940167 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" event={"ID":"55d5dc59-6c63-4519-b489-8f42891ea2be","Type":"ContainerStarted","Data":"201d1dce5a620f107031a0b5c11a10522c2eb49e43f60060264dc6bbd93a71cb"} Jul 07 07:24:32.940309 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:32.940223 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" Jul 07 07:24:32.961430 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:32.961384 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" podStartSLOduration=1.622131314 podStartE2EDuration="1.961369851s" totalImagesPullingTime="339.238537ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:24:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:24:32.958753256 +0000 UTC m=+1528.294719573" watchObservedRunningTime="2026-07-07 07:24:32.961369851 +0000 UTC m=+1528.297336167" Jul 07 07:24:33.628210 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:33.628175 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:34.657421 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:34.657386 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:34.748767 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:34.748737 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:35.726239 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:35.726205 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:35.909230 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:35.909200 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:36.012118 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.012091 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:36.118358 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.118312 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm"] Jul 07 07:24:36.118565 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.118538 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" podUID="55d5dc59-6c63-4519-b489-8f42891ea2be" containerName="maas-api" containerID="cri-o://a0dba73391366f5a8292ad89df9ad9e54ea08a348a447710fc7406aa7a0cdfd3" gracePeriod=30 Jul 07 07:24:36.144035 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.144006 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:36.358783 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.358762 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" Jul 07 07:24:36.472502 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.472470 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55d5dc59-6c63-4519-b489-8f42891ea2be-maas-api-tls\") pod \"55d5dc59-6c63-4519-b489-8f42891ea2be\" (UID: \"55d5dc59-6c63-4519-b489-8f42891ea2be\") " Jul 07 07:24:36.472666 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.472533 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwnkg\" (UniqueName: \"kubernetes.io/projected/55d5dc59-6c63-4519-b489-8f42891ea2be-kube-api-access-kwnkg\") pod \"55d5dc59-6c63-4519-b489-8f42891ea2be\" (UID: \"55d5dc59-6c63-4519-b489-8f42891ea2be\") " Jul 07 07:24:36.472726 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.472704 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55d5dc59-6c63-4519-b489-8f42891ea2be-openshift-service-ca\") pod \"55d5dc59-6c63-4519-b489-8f42891ea2be\" (UID: \"55d5dc59-6c63-4519-b489-8f42891ea2be\") " Jul 07 07:24:36.473009 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.472986 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55d5dc59-6c63-4519-b489-8f42891ea2be-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "55d5dc59-6c63-4519-b489-8f42891ea2be" (UID: "55d5dc59-6c63-4519-b489-8f42891ea2be"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:24:36.474636 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.474607 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55d5dc59-6c63-4519-b489-8f42891ea2be-kube-api-access-kwnkg" (OuterVolumeSpecName: "kube-api-access-kwnkg") pod "55d5dc59-6c63-4519-b489-8f42891ea2be" (UID: "55d5dc59-6c63-4519-b489-8f42891ea2be"). InnerVolumeSpecName "kube-api-access-kwnkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:24:36.474753 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.474649 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55d5dc59-6c63-4519-b489-8f42891ea2be-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "55d5dc59-6c63-4519-b489-8f42891ea2be" (UID: "55d5dc59-6c63-4519-b489-8f42891ea2be"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:24:36.573692 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.573623 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwnkg\" (UniqueName: \"kubernetes.io/projected/55d5dc59-6c63-4519-b489-8f42891ea2be-kube-api-access-kwnkg\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:24:36.573692 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.573649 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55d5dc59-6c63-4519-b489-8f42891ea2be-openshift-service-ca\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:24:36.573692 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.573659 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55d5dc59-6c63-4519-b489-8f42891ea2be-maas-api-tls\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:24:36.959868 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.959781 2573 generic.go:356] "Generic (PLEG): container finished" podID="55d5dc59-6c63-4519-b489-8f42891ea2be" containerID="a0dba73391366f5a8292ad89df9ad9e54ea08a348a447710fc7406aa7a0cdfd3" exitCode=0 Jul 07 07:24:36.959868 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.959826 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" Jul 07 07:24:36.959868 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.959840 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" event={"ID":"55d5dc59-6c63-4519-b489-8f42891ea2be","Type":"ContainerDied","Data":"a0dba73391366f5a8292ad89df9ad9e54ea08a348a447710fc7406aa7a0cdfd3"} Jul 07 07:24:36.960381 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.959881 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm" event={"ID":"55d5dc59-6c63-4519-b489-8f42891ea2be","Type":"ContainerDied","Data":"201d1dce5a620f107031a0b5c11a10522c2eb49e43f60060264dc6bbd93a71cb"} Jul 07 07:24:36.960381 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.959902 2573 scope.go:117] "RemoveContainer" containerID="a0dba73391366f5a8292ad89df9ad9e54ea08a348a447710fc7406aa7a0cdfd3" Jul 07 07:24:36.968791 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.968769 2573 scope.go:117] "RemoveContainer" containerID="a0dba73391366f5a8292ad89df9ad9e54ea08a348a447710fc7406aa7a0cdfd3" Jul 07 07:24:36.969038 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:24:36.969018 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0dba73391366f5a8292ad89df9ad9e54ea08a348a447710fc7406aa7a0cdfd3\": container with ID starting with a0dba73391366f5a8292ad89df9ad9e54ea08a348a447710fc7406aa7a0cdfd3 not found: ID does not exist" containerID="a0dba73391366f5a8292ad89df9ad9e54ea08a348a447710fc7406aa7a0cdfd3" Jul 07 07:24:36.969089 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.969051 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0dba73391366f5a8292ad89df9ad9e54ea08a348a447710fc7406aa7a0cdfd3"} err="failed to get container status \"a0dba73391366f5a8292ad89df9ad9e54ea08a348a447710fc7406aa7a0cdfd3\": rpc error: code = NotFound desc = could not find container \"a0dba73391366f5a8292ad89df9ad9e54ea08a348a447710fc7406aa7a0cdfd3\": container with ID starting with a0dba73391366f5a8292ad89df9ad9e54ea08a348a447710fc7406aa7a0cdfd3 not found: ID does not exist" Jul 07 07:24:36.985135 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.985113 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm"] Jul 07 07:24:36.990749 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:36.990730 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-3b8063ff-69c6964b65-qfwcm"] Jul 07 07:24:37.110461 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:37.110432 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55d5dc59-6c63-4519-b489-8f42891ea2be" path="/var/lib/kubelet/pods/55d5dc59-6c63-4519-b489-8f42891ea2be/volumes" Jul 07 07:24:37.722125 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:37.722098 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:42.341711 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:42.341671 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:42.415224 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:42.415191 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:42.829602 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:42.829557 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:42.996463 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:42.996433 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:43.143391 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:43.143301 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:44.458470 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:44.458436 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:50.749562 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:50.749522 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:50.857980 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:50.857927 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:50.957512 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:50.957477 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:51.059739 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:51.059665 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:51.475467 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:51.475383 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:24:53.644029 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:24:53.643998 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:25:13.208487 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:25:13.208450 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:25:17.480790 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:25:17.480757 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:25:23.772963 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:25:23.772930 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:25:23.876160 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:25:23.876122 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:25:23.972555 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:25:23.972518 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:25:39.907154 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:25:39.907115 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:25:45.858351 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:25:45.858295 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:25:45.957815 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:25:45.957777 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:25:46.057310 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:25:46.057278 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:25:56.554801 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:25:56.554768 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:25:57.330759 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:25:57.330730 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:03.614494 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:03.614455 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:03.687989 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:03.687957 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:03.791857 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:03.791826 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:03.885093 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:03.885015 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:04.734959 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:04.734926 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:04.866192 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:04.866160 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:06.516088 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:06.516057 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:12.634986 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:12.634932 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:12.731970 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:12.731927 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:13.317090 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:13.317059 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:19.815482 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:19.815445 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:19.881956 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:19.881924 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:20.068291 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:20.068207 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:20.381576 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:20.381499 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:20.532191 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:20.532162 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:23.261655 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.261610 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:23.362172 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.362144 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:23.504374 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.504344 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg"] Jul 07 07:26:23.504758 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.504742 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55d5dc59-6c63-4519-b489-8f42891ea2be" containerName="maas-api" Jul 07 07:26:23.504834 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.504760 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="55d5dc59-6c63-4519-b489-8f42891ea2be" containerName="maas-api" Jul 07 07:26:23.504878 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.504866 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="55d5dc59-6c63-4519-b489-8f42891ea2be" containerName="maas-api" Jul 07 07:26:23.507970 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.507953 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" Jul 07 07:26:23.510484 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.510466 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-8305e258-serving-cert\"" Jul 07 07:26:23.519994 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.519964 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg"] Jul 07 07:26:23.545163 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.545135 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:23.610477 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.610443 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8fwm\" (UniqueName: \"kubernetes.io/projected/5d37d200-b102-4684-a817-fae7365c59ce-kube-api-access-b8fwm\") pod \"maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg\" (UID: \"5d37d200-b102-4684-a817-fae7365c59ce\") " pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" Jul 07 07:26:23.610623 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.610554 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d37d200-b102-4684-a817-fae7365c59ce-openshift-service-ca\") pod \"maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg\" (UID: \"5d37d200-b102-4684-a817-fae7365c59ce\") " pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" Jul 07 07:26:23.610691 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.610646 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d37d200-b102-4684-a817-fae7365c59ce-maas-api-tls\") pod \"maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg\" (UID: \"5d37d200-b102-4684-a817-fae7365c59ce\") " pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" Jul 07 07:26:23.711938 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.711911 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8fwm\" (UniqueName: \"kubernetes.io/projected/5d37d200-b102-4684-a817-fae7365c59ce-kube-api-access-b8fwm\") pod \"maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg\" (UID: \"5d37d200-b102-4684-a817-fae7365c59ce\") " pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" Jul 07 07:26:23.712083 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.711950 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d37d200-b102-4684-a817-fae7365c59ce-openshift-service-ca\") pod \"maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg\" (UID: \"5d37d200-b102-4684-a817-fae7365c59ce\") " pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" Jul 07 07:26:23.712083 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.711992 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d37d200-b102-4684-a817-fae7365c59ce-maas-api-tls\") pod \"maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg\" (UID: \"5d37d200-b102-4684-a817-fae7365c59ce\") " pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" Jul 07 07:26:23.712166 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:26:23.712098 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-8305e258-serving-cert: secret "maas-api-e2e-mt-8305e258-serving-cert" not found Jul 07 07:26:23.712166 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:26:23.712152 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d37d200-b102-4684-a817-fae7365c59ce-maas-api-tls podName:5d37d200-b102-4684-a817-fae7365c59ce nodeName:}" failed. No retries permitted until 2026-07-07 07:26:24.212136606 +0000 UTC m=+1639.548102902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5d37d200-b102-4684-a817-fae7365c59ce-maas-api-tls") pod "maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" (UID: "5d37d200-b102-4684-a817-fae7365c59ce") : secret "maas-api-e2e-mt-8305e258-serving-cert" not found Jul 07 07:26:23.712812 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.712789 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d37d200-b102-4684-a817-fae7365c59ce-openshift-service-ca\") pod \"maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg\" (UID: \"5d37d200-b102-4684-a817-fae7365c59ce\") " pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" Jul 07 07:26:23.721313 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.721290 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8fwm\" (UniqueName: \"kubernetes.io/projected/5d37d200-b102-4684-a817-fae7365c59ce-kube-api-access-b8fwm\") pod \"maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg\" (UID: \"5d37d200-b102-4684-a817-fae7365c59ce\") " pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" Jul 07 07:26:23.941670 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:23.941584 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:24.111416 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:24.111374 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:24.210038 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:24.209956 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:24.215984 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:24.215961 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d37d200-b102-4684-a817-fae7365c59ce-maas-api-tls\") pod \"maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg\" (UID: \"5d37d200-b102-4684-a817-fae7365c59ce\") " pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" Jul 07 07:26:24.218404 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:24.218380 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d37d200-b102-4684-a817-fae7365c59ce-maas-api-tls\") pod \"maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg\" (UID: \"5d37d200-b102-4684-a817-fae7365c59ce\") " pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" Jul 07 07:26:24.420101 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:24.420072 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" Jul 07 07:26:24.562794 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:24.562767 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg"] Jul 07 07:26:24.565196 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:26:24.565160 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d37d200_b102_4684_a817_fae7365c59ce.slice/crio-26cb08affd534a768930e0d51c743979deacdf296e16b01bc23184dcba464522 WatchSource:0}: Error finding container 26cb08affd534a768930e0d51c743979deacdf296e16b01bc23184dcba464522: Status 404 returned error can't find the container with id 26cb08affd534a768930e0d51c743979deacdf296e16b01bc23184dcba464522 Jul 07 07:26:25.453985 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:25.453888 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" event={"ID":"5d37d200-b102-4684-a817-fae7365c59ce","Type":"ContainerStarted","Data":"6050809ee9df97f9010ea17326c7dff09297d9facf04af62806288d962261cf7"} Jul 07 07:26:25.453985 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:25.453982 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" event={"ID":"5d37d200-b102-4684-a817-fae7365c59ce","Type":"ContainerStarted","Data":"26cb08affd534a768930e0d51c743979deacdf296e16b01bc23184dcba464522"} Jul 07 07:26:25.454397 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:25.454013 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" Jul 07 07:26:25.475180 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:25.475136 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" podStartSLOduration=1.869875875 podStartE2EDuration="2.475119148s" totalImagesPullingTime="605.243273ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:26:25.473419124 +0000 UTC m=+1640.809385443" watchObservedRunningTime="2026-07-07 07:26:25.475119148 +0000 UTC m=+1640.811085467" Jul 07 07:26:27.654266 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:27.654231 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:28.310383 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:28.310342 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:28.399025 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:28.398992 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:28.750109 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:28.750077 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:31.464651 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:31.464625 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" Jul 07 07:26:34.926390 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:34.926312 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:35.028339 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:35.028293 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:35.407552 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:35.407521 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:35.809187 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:35.809156 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg"] Jul 07 07:26:35.809675 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:35.809641 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" podUID="5d37d200-b102-4684-a817-fae7365c59ce" containerName="maas-api" containerID="cri-o://6050809ee9df97f9010ea17326c7dff09297d9facf04af62806288d962261cf7" gracePeriod=30 Jul 07 07:26:35.829800 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:35.829780 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:35.931154 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:35.931129 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:36.059926 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.059860 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" Jul 07 07:26:36.224492 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.224459 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d37d200-b102-4684-a817-fae7365c59ce-maas-api-tls\") pod \"5d37d200-b102-4684-a817-fae7365c59ce\" (UID: \"5d37d200-b102-4684-a817-fae7365c59ce\") " Jul 07 07:26:36.224664 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.224593 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8fwm\" (UniqueName: \"kubernetes.io/projected/5d37d200-b102-4684-a817-fae7365c59ce-kube-api-access-b8fwm\") pod \"5d37d200-b102-4684-a817-fae7365c59ce\" (UID: \"5d37d200-b102-4684-a817-fae7365c59ce\") " Jul 07 07:26:36.224664 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.224621 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d37d200-b102-4684-a817-fae7365c59ce-openshift-service-ca\") pod \"5d37d200-b102-4684-a817-fae7365c59ce\" (UID: \"5d37d200-b102-4684-a817-fae7365c59ce\") " Jul 07 07:26:36.225002 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.224970 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d37d200-b102-4684-a817-fae7365c59ce-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5d37d200-b102-4684-a817-fae7365c59ce" (UID: "5d37d200-b102-4684-a817-fae7365c59ce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:26:36.226608 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.226582 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d37d200-b102-4684-a817-fae7365c59ce-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5d37d200-b102-4684-a817-fae7365c59ce" (UID: "5d37d200-b102-4684-a817-fae7365c59ce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:26:36.226697 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.226617 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d37d200-b102-4684-a817-fae7365c59ce-kube-api-access-b8fwm" (OuterVolumeSpecName: "kube-api-access-b8fwm") pod "5d37d200-b102-4684-a817-fae7365c59ce" (UID: "5d37d200-b102-4684-a817-fae7365c59ce"). InnerVolumeSpecName "kube-api-access-b8fwm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:26:36.325892 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.325810 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8fwm\" (UniqueName: \"kubernetes.io/projected/5d37d200-b102-4684-a817-fae7365c59ce-kube-api-access-b8fwm\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:26:36.325892 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.325839 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d37d200-b102-4684-a817-fae7365c59ce-openshift-service-ca\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:26:36.325892 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.325849 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d37d200-b102-4684-a817-fae7365c59ce-maas-api-tls\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:26:36.505574 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.505542 2573 generic.go:356] "Generic (PLEG): container finished" podID="5d37d200-b102-4684-a817-fae7365c59ce" containerID="6050809ee9df97f9010ea17326c7dff09297d9facf04af62806288d962261cf7" exitCode=0 Jul 07 07:26:36.505740 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.505585 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" Jul 07 07:26:36.505740 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.505581 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" event={"ID":"5d37d200-b102-4684-a817-fae7365c59ce","Type":"ContainerDied","Data":"6050809ee9df97f9010ea17326c7dff09297d9facf04af62806288d962261cf7"} Jul 07 07:26:36.505740 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.505641 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg" event={"ID":"5d37d200-b102-4684-a817-fae7365c59ce","Type":"ContainerDied","Data":"26cb08affd534a768930e0d51c743979deacdf296e16b01bc23184dcba464522"} Jul 07 07:26:36.505740 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.505670 2573 scope.go:117] "RemoveContainer" containerID="6050809ee9df97f9010ea17326c7dff09297d9facf04af62806288d962261cf7" Jul 07 07:26:36.514711 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.514693 2573 scope.go:117] "RemoveContainer" containerID="6050809ee9df97f9010ea17326c7dff09297d9facf04af62806288d962261cf7" Jul 07 07:26:36.514950 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:26:36.514925 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6050809ee9df97f9010ea17326c7dff09297d9facf04af62806288d962261cf7\": container with ID starting with 6050809ee9df97f9010ea17326c7dff09297d9facf04af62806288d962261cf7 not found: ID does not exist" containerID="6050809ee9df97f9010ea17326c7dff09297d9facf04af62806288d962261cf7" Jul 07 07:26:36.515028 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.514951 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6050809ee9df97f9010ea17326c7dff09297d9facf04af62806288d962261cf7"} err="failed to get container status \"6050809ee9df97f9010ea17326c7dff09297d9facf04af62806288d962261cf7\": rpc error: code = NotFound desc = could not find container \"6050809ee9df97f9010ea17326c7dff09297d9facf04af62806288d962261cf7\": container with ID starting with 6050809ee9df97f9010ea17326c7dff09297d9facf04af62806288d962261cf7 not found: ID does not exist" Jul 07 07:26:36.530544 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.530522 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg"] Jul 07 07:26:36.535114 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:36.535095 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-8305e258-5bcc79d45c-qb6gg"] Jul 07 07:26:37.108183 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:37.108153 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d37d200-b102-4684-a817-fae7365c59ce" path="/var/lib/kubelet/pods/5d37d200-b102-4684-a817-fae7365c59ce/volumes" Jul 07 07:26:42.264302 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:42.264264 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:42.364437 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:42.364382 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:42.542821 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:42.542737 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:42.770251 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:42.770223 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:42.902807 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:42.902727 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:43.002098 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:43.002072 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:43.099586 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:43.099547 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:49.459413 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:49.459362 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:49.570693 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:49.570657 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:49.684831 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:49.684797 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:49.798282 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:49.798244 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:56.009220 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:56.009188 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:26:56.096156 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:26:56.096122 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:02.780742 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:02.780707 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:03.109036 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:03.108952 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:03.409508 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:03.409415 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:06.439694 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.439595 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:06.518938 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.518909 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn"] Jul 07 07:27:06.519285 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.519273 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d37d200-b102-4684-a817-fae7365c59ce" containerName="maas-api" Jul 07 07:27:06.519343 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.519287 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d37d200-b102-4684-a817-fae7365c59ce" containerName="maas-api" Jul 07 07:27:06.519384 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.519359 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d37d200-b102-4684-a817-fae7365c59ce" containerName="maas-api" Jul 07 07:27:06.522298 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.522281 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" Jul 07 07:27:06.525197 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.525173 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-18ebf0cb-serving-cert\"" Jul 07 07:27:06.537173 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.537150 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn"] Jul 07 07:27:06.542906 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.542884 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:06.558224 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.558201 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/285937ab-face-4187-a926-f788e3734ea1-maas-api-tls\") pod \"maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn\" (UID: \"285937ab-face-4187-a926-f788e3734ea1\") " pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" Jul 07 07:27:06.558318 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.558255 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28jdc\" (UniqueName: \"kubernetes.io/projected/285937ab-face-4187-a926-f788e3734ea1-kube-api-access-28jdc\") pod \"maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn\" (UID: \"285937ab-face-4187-a926-f788e3734ea1\") " pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" Jul 07 07:27:06.558318 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.558282 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/285937ab-face-4187-a926-f788e3734ea1-openshift-service-ca\") pod \"maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn\" (UID: \"285937ab-face-4187-a926-f788e3734ea1\") " pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" Jul 07 07:27:06.638602 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.638571 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:06.658825 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.658797 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28jdc\" (UniqueName: \"kubernetes.io/projected/285937ab-face-4187-a926-f788e3734ea1-kube-api-access-28jdc\") pod \"maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn\" (UID: \"285937ab-face-4187-a926-f788e3734ea1\") " pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" Jul 07 07:27:06.658969 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.658833 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/285937ab-face-4187-a926-f788e3734ea1-openshift-service-ca\") pod \"maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn\" (UID: \"285937ab-face-4187-a926-f788e3734ea1\") " pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" Jul 07 07:27:06.659042 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.659023 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/285937ab-face-4187-a926-f788e3734ea1-maas-api-tls\") pod \"maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn\" (UID: \"285937ab-face-4187-a926-f788e3734ea1\") " pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" Jul 07 07:27:06.659426 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.659409 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/285937ab-face-4187-a926-f788e3734ea1-openshift-service-ca\") pod \"maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn\" (UID: \"285937ab-face-4187-a926-f788e3734ea1\") " pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" Jul 07 07:27:06.661334 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.661308 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/285937ab-face-4187-a926-f788e3734ea1-maas-api-tls\") pod \"maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn\" (UID: \"285937ab-face-4187-a926-f788e3734ea1\") " pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" Jul 07 07:27:06.685026 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.684995 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28jdc\" (UniqueName: \"kubernetes.io/projected/285937ab-face-4187-a926-f788e3734ea1-kube-api-access-28jdc\") pod \"maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn\" (UID: \"285937ab-face-4187-a926-f788e3734ea1\") " pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" Jul 07 07:27:06.742998 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.742923 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:06.832667 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.832634 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" Jul 07 07:27:06.973689 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:06.973669 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn"] Jul 07 07:27:06.975840 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:27:06.975814 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod285937ab_face_4187_a926_f788e3734ea1.slice/crio-0df7968543e07790c7a965a285e1ec79318d42b2baa783c0cc5c1b290763e9cf WatchSource:0}: Error finding container 0df7968543e07790c7a965a285e1ec79318d42b2baa783c0cc5c1b290763e9cf: Status 404 returned error can't find the container with id 0df7968543e07790c7a965a285e1ec79318d42b2baa783c0cc5c1b290763e9cf Jul 07 07:27:07.055199 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:07.055168 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:07.336872 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:07.336770 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:07.649277 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:07.649193 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" event={"ID":"285937ab-face-4187-a926-f788e3734ea1","Type":"ContainerStarted","Data":"8c1a7e4d939d668be1354c042e0160f7734cd0701738238260339bd38d815301"} Jul 07 07:27:07.649277 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:07.649255 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" event={"ID":"285937ab-face-4187-a926-f788e3734ea1","Type":"ContainerStarted","Data":"0df7968543e07790c7a965a285e1ec79318d42b2baa783c0cc5c1b290763e9cf"} Jul 07 07:27:07.649674 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:07.649288 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" Jul 07 07:27:07.669064 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:07.669020 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" podStartSLOduration=1.162790862 podStartE2EDuration="1.669005645s" totalImagesPullingTime="506.214783ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:27:07.667954968 +0000 UTC m=+1683.003921306" watchObservedRunningTime="2026-07-07 07:27:07.669005645 +0000 UTC m=+1683.004971962" Jul 07 07:27:10.942368 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:10.942310 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:13.659191 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:13.659159 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" Jul 07 07:27:19.590295 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:19.590246 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:40.388565 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:40.388513 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:40.488112 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:40.488024 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:54.125475 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:54.125413 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:54.593637 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:54.593567 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:54.696016 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:54.695962 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:54.832355 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:54.832293 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn"] Jul 07 07:27:54.832709 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:54.832613 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" podUID="285937ab-face-4187-a926-f788e3734ea1" containerName="maas-api" containerID="cri-o://8c1a7e4d939d668be1354c042e0160f7734cd0701738238260339bd38d815301" gracePeriod=30 Jul 07 07:27:54.864067 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:54.864041 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:27:55.087093 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.087072 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" Jul 07 07:27:55.185256 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.185177 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/285937ab-face-4187-a926-f788e3734ea1-openshift-service-ca\") pod \"285937ab-face-4187-a926-f788e3734ea1\" (UID: \"285937ab-face-4187-a926-f788e3734ea1\") " Jul 07 07:27:55.185256 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.185219 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/285937ab-face-4187-a926-f788e3734ea1-maas-api-tls\") pod \"285937ab-face-4187-a926-f788e3734ea1\" (UID: \"285937ab-face-4187-a926-f788e3734ea1\") " Jul 07 07:27:55.185256 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.185236 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28jdc\" (UniqueName: \"kubernetes.io/projected/285937ab-face-4187-a926-f788e3734ea1-kube-api-access-28jdc\") pod \"285937ab-face-4187-a926-f788e3734ea1\" (UID: \"285937ab-face-4187-a926-f788e3734ea1\") " Jul 07 07:27:55.185791 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.185608 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/285937ab-face-4187-a926-f788e3734ea1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "285937ab-face-4187-a926-f788e3734ea1" (UID: "285937ab-face-4187-a926-f788e3734ea1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:27:55.187292 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.187265 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/285937ab-face-4187-a926-f788e3734ea1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "285937ab-face-4187-a926-f788e3734ea1" (UID: "285937ab-face-4187-a926-f788e3734ea1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:27:55.187421 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.187292 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/285937ab-face-4187-a926-f788e3734ea1-kube-api-access-28jdc" (OuterVolumeSpecName: "kube-api-access-28jdc") pod "285937ab-face-4187-a926-f788e3734ea1" (UID: "285937ab-face-4187-a926-f788e3734ea1"). InnerVolumeSpecName "kube-api-access-28jdc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:27:55.286130 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.286098 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/285937ab-face-4187-a926-f788e3734ea1-openshift-service-ca\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:27:55.286130 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.286123 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/285937ab-face-4187-a926-f788e3734ea1-maas-api-tls\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:27:55.286130 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.286132 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28jdc\" (UniqueName: \"kubernetes.io/projected/285937ab-face-4187-a926-f788e3734ea1-kube-api-access-28jdc\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:27:55.861794 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.861755 2573 generic.go:356] "Generic (PLEG): container finished" podID="285937ab-face-4187-a926-f788e3734ea1" containerID="8c1a7e4d939d668be1354c042e0160f7734cd0701738238260339bd38d815301" exitCode=0 Jul 07 07:27:55.861952 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.861818 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" Jul 07 07:27:55.861952 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.861839 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" event={"ID":"285937ab-face-4187-a926-f788e3734ea1","Type":"ContainerDied","Data":"8c1a7e4d939d668be1354c042e0160f7734cd0701738238260339bd38d815301"} Jul 07 07:27:55.861952 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.861912 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn" event={"ID":"285937ab-face-4187-a926-f788e3734ea1","Type":"ContainerDied","Data":"0df7968543e07790c7a965a285e1ec79318d42b2baa783c0cc5c1b290763e9cf"} Jul 07 07:27:55.861952 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.861934 2573 scope.go:117] "RemoveContainer" containerID="8c1a7e4d939d668be1354c042e0160f7734cd0701738238260339bd38d815301" Jul 07 07:27:55.871765 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.871744 2573 scope.go:117] "RemoveContainer" containerID="8c1a7e4d939d668be1354c042e0160f7734cd0701738238260339bd38d815301" Jul 07 07:27:55.872045 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:27:55.872024 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c1a7e4d939d668be1354c042e0160f7734cd0701738238260339bd38d815301\": container with ID starting with 8c1a7e4d939d668be1354c042e0160f7734cd0701738238260339bd38d815301 not found: ID does not exist" containerID="8c1a7e4d939d668be1354c042e0160f7734cd0701738238260339bd38d815301" Jul 07 07:27:55.872127 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.872055 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c1a7e4d939d668be1354c042e0160f7734cd0701738238260339bd38d815301"} err="failed to get container status \"8c1a7e4d939d668be1354c042e0160f7734cd0701738238260339bd38d815301\": rpc error: code = NotFound desc = could not find container \"8c1a7e4d939d668be1354c042e0160f7734cd0701738238260339bd38d815301\": container with ID starting with 8c1a7e4d939d668be1354c042e0160f7734cd0701738238260339bd38d815301 not found: ID does not exist" Jul 07 07:27:55.886789 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.886762 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn"] Jul 07 07:27:55.893694 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:55.893669 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-18ebf0cb-575b8f9b4b-d5fpn"] Jul 07 07:27:57.108053 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:27:57.108012 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="285937ab-face-4187-a926-f788e3734ea1" path="/var/lib/kubelet/pods/285937ab-face-4187-a926-f788e3734ea1/volumes" Jul 07 07:28:00.187295 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:00.187261 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:28:05.225548 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:05.225508 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:28:41.803664 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:41.803634 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:28:42.817472 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:42.817435 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:28:48.932269 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:48.932233 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:28:49.033697 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:49.033664 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:28:49.212267 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:49.212191 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:28:49.441771 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:49.441737 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:28:50.500914 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:50.500839 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:28:50.594232 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:50.594200 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:28:50.697045 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:50.697014 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:28:51.417834 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:51.417805 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:28:51.516708 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:51.516674 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:28:51.616601 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:51.616572 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:28:53.009515 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:53.009481 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:28:59.624496 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:59.624405 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:28:59.711156 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:28:59.711125 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:00.328032 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:00.327999 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:05.157297 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:05.157267 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:29:05.163822 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:05.163802 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:29:06.435307 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:06.435274 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:06.502822 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:06.502793 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:06.590814 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:06.590780 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:06.695230 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:06.695154 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:17.138725 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:17.138663 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:17.489351 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:17.489256 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:38.427889 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:38.427852 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:44.457390 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:44.457346 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:44.533158 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:44.533123 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:44.820882 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:44.820851 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:45.146018 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:45.145944 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:45.494247 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:45.494216 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:48.377270 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.377229 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:48.453938 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.453906 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb"] Jul 07 07:29:48.454463 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.454445 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="285937ab-face-4187-a926-f788e3734ea1" containerName="maas-api" Jul 07 07:29:48.454463 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.454464 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="285937ab-face-4187-a926-f788e3734ea1" containerName="maas-api" Jul 07 07:29:48.454600 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.454577 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="285937ab-face-4187-a926-f788e3734ea1" containerName="maas-api" Jul 07 07:29:48.457904 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.457885 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" Jul 07 07:29:48.460582 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.460557 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-ebf5a0-serving-cert\"" Jul 07 07:29:48.470945 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.470925 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb"] Jul 07 07:29:48.473848 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.473824 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:48.542646 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.542620 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:48.584392 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.584356 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df2f030a-188d-46bf-b1dd-da7a5f229eac-maas-api-tls\") pod \"maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb\" (UID: \"df2f030a-188d-46bf-b1dd-da7a5f229eac\") " pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" Jul 07 07:29:48.584545 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.584447 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjw6r\" (UniqueName: \"kubernetes.io/projected/df2f030a-188d-46bf-b1dd-da7a5f229eac-kube-api-access-qjw6r\") pod \"maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb\" (UID: \"df2f030a-188d-46bf-b1dd-da7a5f229eac\") " pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" Jul 07 07:29:48.584545 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.584474 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df2f030a-188d-46bf-b1dd-da7a5f229eac-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb\" (UID: \"df2f030a-188d-46bf-b1dd-da7a5f229eac\") " pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" Jul 07 07:29:48.647673 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.647591 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:48.685294 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.685259 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjw6r\" (UniqueName: \"kubernetes.io/projected/df2f030a-188d-46bf-b1dd-da7a5f229eac-kube-api-access-qjw6r\") pod \"maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb\" (UID: \"df2f030a-188d-46bf-b1dd-da7a5f229eac\") " pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" Jul 07 07:29:48.685476 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.685312 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df2f030a-188d-46bf-b1dd-da7a5f229eac-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb\" (UID: \"df2f030a-188d-46bf-b1dd-da7a5f229eac\") " pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" Jul 07 07:29:48.685476 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.685373 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df2f030a-188d-46bf-b1dd-da7a5f229eac-maas-api-tls\") pod \"maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb\" (UID: \"df2f030a-188d-46bf-b1dd-da7a5f229eac\") " pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" Jul 07 07:29:48.686001 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.685983 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df2f030a-188d-46bf-b1dd-da7a5f229eac-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb\" (UID: \"df2f030a-188d-46bf-b1dd-da7a5f229eac\") " pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" Jul 07 07:29:48.687780 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.687762 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df2f030a-188d-46bf-b1dd-da7a5f229eac-maas-api-tls\") pod \"maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb\" (UID: \"df2f030a-188d-46bf-b1dd-da7a5f229eac\") " pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" Jul 07 07:29:48.693888 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.693870 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjw6r\" (UniqueName: \"kubernetes.io/projected/df2f030a-188d-46bf-b1dd-da7a5f229eac-kube-api-access-qjw6r\") pod \"maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb\" (UID: \"df2f030a-188d-46bf-b1dd-da7a5f229eac\") " pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" Jul 07 07:29:48.768712 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.768677 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" Jul 07 07:29:48.922131 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.922107 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb"] Jul 07 07:29:48.924233 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:29:48.924205 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf2f030a_188d_46bf_b1dd_da7a5f229eac.slice/crio-a43696797a1945c250e2e72c940d70217d79decfe94bf32cf2b4f5af814cc4ff WatchSource:0}: Error finding container a43696797a1945c250e2e72c940d70217d79decfe94bf32cf2b4f5af814cc4ff: Status 404 returned error can't find the container with id a43696797a1945c250e2e72c940d70217d79decfe94bf32cf2b4f5af814cc4ff Jul 07 07:29:48.925902 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:48.925885 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 07:29:49.020308 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:49.020278 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:49.121753 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:49.121723 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:49.381568 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:49.381543 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" event={"ID":"df2f030a-188d-46bf-b1dd-da7a5f229eac","Type":"ContainerStarted","Data":"a43696797a1945c250e2e72c940d70217d79decfe94bf32cf2b4f5af814cc4ff"} Jul 07 07:29:50.388587 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:50.388556 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" event={"ID":"df2f030a-188d-46bf-b1dd-da7a5f229eac","Type":"ContainerStarted","Data":"d78fc3eb254aea32c73c33d65657725839469a4730f26da91aa2dcbf481229c1"} Jul 07 07:29:50.388983 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:50.388672 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" Jul 07 07:29:50.408560 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:50.408512 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" podStartSLOduration=2.019434179 podStartE2EDuration="2.408497512s" totalImagesPullingTime="389.063333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:29:50.406784412 +0000 UTC m=+1845.742750729" watchObservedRunningTime="2026-07-07 07:29:50.408497512 +0000 UTC m=+1845.744463829" Jul 07 07:29:52.861007 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:52.860977 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:53.200951 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:53.200872 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:53.581994 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:53.581963 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:54.437273 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:54.437239 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:56.396579 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.396533 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:56.397933 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.397911 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" Jul 07 07:29:56.497737 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.497705 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:56.590600 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.590570 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45"] Jul 07 07:29:56.594405 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.594374 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" Jul 07 07:29:56.597097 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.597074 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-e309b3-serving-cert\"" Jul 07 07:29:56.606993 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.606975 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45"] Jul 07 07:29:56.639015 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.638988 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:56.660295 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.660229 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d59078d6-d421-4e57-83b9-40af8d6ab70e-maas-api-tls\") pod \"maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45\" (UID: \"d59078d6-d421-4e57-83b9-40af8d6ab70e\") " pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" Jul 07 07:29:56.660423 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.660312 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d59078d6-d421-4e57-83b9-40af8d6ab70e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45\" (UID: \"d59078d6-d421-4e57-83b9-40af8d6ab70e\") " pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" Jul 07 07:29:56.660483 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.660411 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsbww\" (UniqueName: \"kubernetes.io/projected/d59078d6-d421-4e57-83b9-40af8d6ab70e-kube-api-access-tsbww\") pod \"maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45\" (UID: \"d59078d6-d421-4e57-83b9-40af8d6ab70e\") " pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" Jul 07 07:29:56.761413 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.761380 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d59078d6-d421-4e57-83b9-40af8d6ab70e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45\" (UID: \"d59078d6-d421-4e57-83b9-40af8d6ab70e\") " pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" Jul 07 07:29:56.761573 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.761441 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsbww\" (UniqueName: \"kubernetes.io/projected/d59078d6-d421-4e57-83b9-40af8d6ab70e-kube-api-access-tsbww\") pod \"maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45\" (UID: \"d59078d6-d421-4e57-83b9-40af8d6ab70e\") " pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" Jul 07 07:29:56.761573 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.761476 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d59078d6-d421-4e57-83b9-40af8d6ab70e-maas-api-tls\") pod \"maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45\" (UID: \"d59078d6-d421-4e57-83b9-40af8d6ab70e\") " pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" Jul 07 07:29:56.761702 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:29:56.761606 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-e309b3-serving-cert: secret "maas-api-e2e-inf-b-e309b3-serving-cert" not found Jul 07 07:29:56.761702 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:29:56.761678 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d59078d6-d421-4e57-83b9-40af8d6ab70e-maas-api-tls podName:d59078d6-d421-4e57-83b9-40af8d6ab70e nodeName:}" failed. No retries permitted until 2026-07-07 07:29:57.261659716 +0000 UTC m=+1852.597626014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d59078d6-d421-4e57-83b9-40af8d6ab70e-maas-api-tls") pod "maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" (UID: "d59078d6-d421-4e57-83b9-40af8d6ab70e") : secret "maas-api-e2e-inf-b-e309b3-serving-cert" not found Jul 07 07:29:56.762080 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.762057 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d59078d6-d421-4e57-83b9-40af8d6ab70e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45\" (UID: \"d59078d6-d421-4e57-83b9-40af8d6ab70e\") " pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" Jul 07 07:29:56.771690 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.771667 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsbww\" (UniqueName: \"kubernetes.io/projected/d59078d6-d421-4e57-83b9-40af8d6ab70e-kube-api-access-tsbww\") pod \"maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45\" (UID: \"d59078d6-d421-4e57-83b9-40af8d6ab70e\") " pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" Jul 07 07:29:56.906140 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:56.906107 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:57.189255 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:57.189220 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:57.264720 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:57.264681 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d59078d6-d421-4e57-83b9-40af8d6ab70e-maas-api-tls\") pod \"maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45\" (UID: \"d59078d6-d421-4e57-83b9-40af8d6ab70e\") " pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" Jul 07 07:29:57.267055 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:57.267034 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d59078d6-d421-4e57-83b9-40af8d6ab70e-maas-api-tls\") pod \"maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45\" (UID: \"d59078d6-d421-4e57-83b9-40af8d6ab70e\") " pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" Jul 07 07:29:57.290078 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:57.290056 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:29:57.508025 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:57.507994 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" Jul 07 07:29:57.646852 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:57.646827 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45"] Jul 07 07:29:57.648790 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:29:57.648766 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd59078d6_d421_4e57_83b9_40af8d6ab70e.slice/crio-2c8df3fd22a4eb0a79183dfc077626f29af202e57bae98301cc6c719068636b4 WatchSource:0}: Error finding container 2c8df3fd22a4eb0a79183dfc077626f29af202e57bae98301cc6c719068636b4: Status 404 returned error can't find the container with id 2c8df3fd22a4eb0a79183dfc077626f29af202e57bae98301cc6c719068636b4 Jul 07 07:29:58.427012 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:58.426928 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" event={"ID":"d59078d6-d421-4e57-83b9-40af8d6ab70e","Type":"ContainerStarted","Data":"d152cd08ec47116a9b8cd633e97341e5b866149122b24424c7f366e083c5d2a5"} Jul 07 07:29:58.427012 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:58.426988 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" event={"ID":"d59078d6-d421-4e57-83b9-40af8d6ab70e","Type":"ContainerStarted","Data":"2c8df3fd22a4eb0a79183dfc077626f29af202e57bae98301cc6c719068636b4"} Jul 07 07:29:58.427195 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:58.427068 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" Jul 07 07:29:58.448609 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:29:58.448565 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" podStartSLOduration=2.053051411 podStartE2EDuration="2.448548805s" totalImagesPullingTime="395.497394ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:29:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:29:58.446600123 +0000 UTC m=+1853.782566440" watchObservedRunningTime="2026-07-07 07:29:58.448548805 +0000 UTC m=+1853.784515122" Jul 07 07:30:00.885031 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:30:00.884996 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:30:04.437549 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:30:04.437521 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" Jul 07 07:30:10.896568 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:30:10.896533 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:30:10.994625 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:30:10.994592 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:30:17.757108 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:30:17.757079 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:30:18.121750 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:30:18.121659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:30:18.221629 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:30:18.221596 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:30:37.133768 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:30:37.133737 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:31:04.331435 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:04.331387 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:31:15.178711 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:15.178679 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:31:15.280041 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:15.280005 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:31:25.326067 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:25.326036 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:31:26.251770 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:26.251738 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:31:26.408961 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:26.408934 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:31:26.814454 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:26.814423 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:31:27.030680 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.030650 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:31:27.132254 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.132173 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:31:27.278033 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.277999 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb"] Jul 07 07:31:27.278224 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.278198 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" podUID="df2f030a-188d-46bf-b1dd-da7a5f229eac" containerName="maas-api" containerID="cri-o://d78fc3eb254aea32c73c33d65657725839469a4730f26da91aa2dcbf481229c1" gracePeriod=30 Jul 07 07:31:27.320796 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.320775 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:31:27.526819 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.526795 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" Jul 07 07:31:27.556734 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.556706 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df2f030a-188d-46bf-b1dd-da7a5f229eac-maas-api-tls\") pod \"df2f030a-188d-46bf-b1dd-da7a5f229eac\" (UID: \"df2f030a-188d-46bf-b1dd-da7a5f229eac\") " Jul 07 07:31:27.556860 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.556814 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df2f030a-188d-46bf-b1dd-da7a5f229eac-openshift-service-ca\") pod \"df2f030a-188d-46bf-b1dd-da7a5f229eac\" (UID: \"df2f030a-188d-46bf-b1dd-da7a5f229eac\") " Jul 07 07:31:27.556860 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.556846 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjw6r\" (UniqueName: \"kubernetes.io/projected/df2f030a-188d-46bf-b1dd-da7a5f229eac-kube-api-access-qjw6r\") pod \"df2f030a-188d-46bf-b1dd-da7a5f229eac\" (UID: \"df2f030a-188d-46bf-b1dd-da7a5f229eac\") " Jul 07 07:31:27.557156 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.557134 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df2f030a-188d-46bf-b1dd-da7a5f229eac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "df2f030a-188d-46bf-b1dd-da7a5f229eac" (UID: "df2f030a-188d-46bf-b1dd-da7a5f229eac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:31:27.558647 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.558618 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df2f030a-188d-46bf-b1dd-da7a5f229eac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "df2f030a-188d-46bf-b1dd-da7a5f229eac" (UID: "df2f030a-188d-46bf-b1dd-da7a5f229eac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:31:27.558835 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.558814 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df2f030a-188d-46bf-b1dd-da7a5f229eac-kube-api-access-qjw6r" (OuterVolumeSpecName: "kube-api-access-qjw6r") pod "df2f030a-188d-46bf-b1dd-da7a5f229eac" (UID: "df2f030a-188d-46bf-b1dd-da7a5f229eac"). InnerVolumeSpecName "kube-api-access-qjw6r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:31:27.657553 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.657495 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df2f030a-188d-46bf-b1dd-da7a5f229eac-maas-api-tls\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:31:27.657553 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.657517 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df2f030a-188d-46bf-b1dd-da7a5f229eac-openshift-service-ca\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:31:27.657553 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.657527 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjw6r\" (UniqueName: \"kubernetes.io/projected/df2f030a-188d-46bf-b1dd-da7a5f229eac-kube-api-access-qjw6r\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:31:27.845943 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.845909 2573 generic.go:356] "Generic (PLEG): container finished" podID="df2f030a-188d-46bf-b1dd-da7a5f229eac" containerID="d78fc3eb254aea32c73c33d65657725839469a4730f26da91aa2dcbf481229c1" exitCode=0 Jul 07 07:31:27.846097 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.845953 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" Jul 07 07:31:27.846097 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.845996 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" event={"ID":"df2f030a-188d-46bf-b1dd-da7a5f229eac","Type":"ContainerDied","Data":"d78fc3eb254aea32c73c33d65657725839469a4730f26da91aa2dcbf481229c1"} Jul 07 07:31:27.846097 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.846044 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb" event={"ID":"df2f030a-188d-46bf-b1dd-da7a5f229eac","Type":"ContainerDied","Data":"a43696797a1945c250e2e72c940d70217d79decfe94bf32cf2b4f5af814cc4ff"} Jul 07 07:31:27.846097 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.846066 2573 scope.go:117] "RemoveContainer" containerID="d78fc3eb254aea32c73c33d65657725839469a4730f26da91aa2dcbf481229c1" Jul 07 07:31:27.855206 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.855184 2573 scope.go:117] "RemoveContainer" containerID="d78fc3eb254aea32c73c33d65657725839469a4730f26da91aa2dcbf481229c1" Jul 07 07:31:27.855453 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:31:27.855434 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d78fc3eb254aea32c73c33d65657725839469a4730f26da91aa2dcbf481229c1\": container with ID starting with d78fc3eb254aea32c73c33d65657725839469a4730f26da91aa2dcbf481229c1 not found: ID does not exist" containerID="d78fc3eb254aea32c73c33d65657725839469a4730f26da91aa2dcbf481229c1" Jul 07 07:31:27.855520 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.855462 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d78fc3eb254aea32c73c33d65657725839469a4730f26da91aa2dcbf481229c1"} err="failed to get container status \"d78fc3eb254aea32c73c33d65657725839469a4730f26da91aa2dcbf481229c1\": rpc error: code = NotFound desc = could not find container \"d78fc3eb254aea32c73c33d65657725839469a4730f26da91aa2dcbf481229c1\": container with ID starting with d78fc3eb254aea32c73c33d65657725839469a4730f26da91aa2dcbf481229c1 not found: ID does not exist" Jul 07 07:31:27.871413 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.871376 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb"] Jul 07 07:31:27.876028 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:27.876008 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ebf5a0-fc69c6898-8czlb"] Jul 07 07:31:29.108004 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:29.107972 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df2f030a-188d-46bf-b1dd-da7a5f229eac" path="/var/lib/kubelet/pods/df2f030a-188d-46bf-b1dd-da7a5f229eac/volumes" Jul 07 07:31:31.873827 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:31:31.873796 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:06.802097 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:06.802061 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:06.863012 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:06.862979 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:08.995366 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:08.995313 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:09.266793 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:09.266714 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:09.508430 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:09.508392 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:09.709381 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:09.709297 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:10.175032 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:10.174999 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:10.300283 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:10.300250 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:10.387197 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:10.387163 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:10.530215 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:10.530182 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45"] Jul 07 07:32:10.530484 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:10.530449 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" podUID="d59078d6-d421-4e57-83b9-40af8d6ab70e" containerName="maas-api" containerID="cri-o://d152cd08ec47116a9b8cd633e97341e5b866149122b24424c7f366e083c5d2a5" gracePeriod=30 Jul 07 07:32:10.561422 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:10.561399 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:10.777894 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:10.777871 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" Jul 07 07:32:10.837023 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:10.836942 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d59078d6-d421-4e57-83b9-40af8d6ab70e-maas-api-tls\") pod \"d59078d6-d421-4e57-83b9-40af8d6ab70e\" (UID: \"d59078d6-d421-4e57-83b9-40af8d6ab70e\") " Jul 07 07:32:10.837023 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:10.837002 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d59078d6-d421-4e57-83b9-40af8d6ab70e-openshift-service-ca\") pod \"d59078d6-d421-4e57-83b9-40af8d6ab70e\" (UID: \"d59078d6-d421-4e57-83b9-40af8d6ab70e\") " Jul 07 07:32:10.837215 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:10.837093 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsbww\" (UniqueName: \"kubernetes.io/projected/d59078d6-d421-4e57-83b9-40af8d6ab70e-kube-api-access-tsbww\") pod \"d59078d6-d421-4e57-83b9-40af8d6ab70e\" (UID: \"d59078d6-d421-4e57-83b9-40af8d6ab70e\") " Jul 07 07:32:10.837392 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:10.837363 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d59078d6-d421-4e57-83b9-40af8d6ab70e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d59078d6-d421-4e57-83b9-40af8d6ab70e" (UID: "d59078d6-d421-4e57-83b9-40af8d6ab70e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:32:10.839118 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:10.839078 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d59078d6-d421-4e57-83b9-40af8d6ab70e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d59078d6-d421-4e57-83b9-40af8d6ab70e" (UID: "d59078d6-d421-4e57-83b9-40af8d6ab70e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:32:10.839504 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:10.839474 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d59078d6-d421-4e57-83b9-40af8d6ab70e-kube-api-access-tsbww" (OuterVolumeSpecName: "kube-api-access-tsbww") pod "d59078d6-d421-4e57-83b9-40af8d6ab70e" (UID: "d59078d6-d421-4e57-83b9-40af8d6ab70e"). InnerVolumeSpecName "kube-api-access-tsbww". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:32:10.938472 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:10.938446 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsbww\" (UniqueName: \"kubernetes.io/projected/d59078d6-d421-4e57-83b9-40af8d6ab70e-kube-api-access-tsbww\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:32:10.938472 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:10.938473 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d59078d6-d421-4e57-83b9-40af8d6ab70e-maas-api-tls\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:32:10.938648 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:10.938485 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d59078d6-d421-4e57-83b9-40af8d6ab70e-openshift-service-ca\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:32:11.049495 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:11.049467 2573 generic.go:356] "Generic (PLEG): container finished" podID="d59078d6-d421-4e57-83b9-40af8d6ab70e" containerID="d152cd08ec47116a9b8cd633e97341e5b866149122b24424c7f366e083c5d2a5" exitCode=0 Jul 07 07:32:11.049675 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:11.049512 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" Jul 07 07:32:11.049675 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:11.049558 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" event={"ID":"d59078d6-d421-4e57-83b9-40af8d6ab70e","Type":"ContainerDied","Data":"d152cd08ec47116a9b8cd633e97341e5b866149122b24424c7f366e083c5d2a5"} Jul 07 07:32:11.049675 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:11.049608 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45" event={"ID":"d59078d6-d421-4e57-83b9-40af8d6ab70e","Type":"ContainerDied","Data":"2c8df3fd22a4eb0a79183dfc077626f29af202e57bae98301cc6c719068636b4"} Jul 07 07:32:11.049675 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:11.049630 2573 scope.go:117] "RemoveContainer" containerID="d152cd08ec47116a9b8cd633e97341e5b866149122b24424c7f366e083c5d2a5" Jul 07 07:32:11.058955 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:11.058934 2573 scope.go:117] "RemoveContainer" containerID="d152cd08ec47116a9b8cd633e97341e5b866149122b24424c7f366e083c5d2a5" Jul 07 07:32:11.059204 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:32:11.059180 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d152cd08ec47116a9b8cd633e97341e5b866149122b24424c7f366e083c5d2a5\": container with ID starting with d152cd08ec47116a9b8cd633e97341e5b866149122b24424c7f366e083c5d2a5 not found: ID does not exist" containerID="d152cd08ec47116a9b8cd633e97341e5b866149122b24424c7f366e083c5d2a5" Jul 07 07:32:11.059287 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:11.059217 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d152cd08ec47116a9b8cd633e97341e5b866149122b24424c7f366e083c5d2a5"} err="failed to get container status \"d152cd08ec47116a9b8cd633e97341e5b866149122b24424c7f366e083c5d2a5\": rpc error: code = NotFound desc = could not find container \"d152cd08ec47116a9b8cd633e97341e5b866149122b24424c7f366e083c5d2a5\": container with ID starting with d152cd08ec47116a9b8cd633e97341e5b866149122b24424c7f366e083c5d2a5 not found: ID does not exist" Jul 07 07:32:11.071929 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:11.071904 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45"] Jul 07 07:32:11.080281 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:11.080258 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e309b3-77dbd557bb-tps45"] Jul 07 07:32:11.108178 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:11.108110 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d59078d6-d421-4e57-83b9-40af8d6ab70e" path="/var/lib/kubelet/pods/d59078d6-d421-4e57-83b9-40af8d6ab70e/volumes" Jul 07 07:32:16.486115 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:16.486080 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:16.587160 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:16.587128 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:16.760072 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:16.760046 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:17.009624 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:17.009592 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:17.413079 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:17.413047 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:17.687618 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:17.687511 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:18.034232 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:18.034196 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:20.317790 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.317755 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:20.390561 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.390522 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw"] Jul 07 07:32:20.391117 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.391084 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d59078d6-d421-4e57-83b9-40af8d6ab70e" containerName="maas-api" Jul 07 07:32:20.391117 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.391113 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d59078d6-d421-4e57-83b9-40af8d6ab70e" containerName="maas-api" Jul 07 07:32:20.391310 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.391143 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df2f030a-188d-46bf-b1dd-da7a5f229eac" containerName="maas-api" Jul 07 07:32:20.391310 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.391154 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="df2f030a-188d-46bf-b1dd-da7a5f229eac" containerName="maas-api" Jul 07 07:32:20.391310 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.391241 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d59078d6-d421-4e57-83b9-40af8d6ab70e" containerName="maas-api" Jul 07 07:32:20.391310 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.391257 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="df2f030a-188d-46bf-b1dd-da7a5f229eac" containerName="maas-api" Jul 07 07:32:20.395674 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.395656 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" Jul 07 07:32:20.396697 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.396671 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:20.398108 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.398087 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-14bf96-serving-cert\"" Jul 07 07:32:20.406715 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.406621 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw"] Jul 07 07:32:20.422290 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.422263 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebc3eceb-c294-4703-a15a-b1fe4de3376a-openshift-service-ca\") pod \"maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw\" (UID: \"ebc3eceb-c294-4703-a15a-b1fe4de3376a\") " pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" Jul 07 07:32:20.422458 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.422300 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6z9g\" (UniqueName: \"kubernetes.io/projected/ebc3eceb-c294-4703-a15a-b1fe4de3376a-kube-api-access-g6z9g\") pod \"maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw\" (UID: \"ebc3eceb-c294-4703-a15a-b1fe4de3376a\") " pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" Jul 07 07:32:20.422458 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.422317 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ebc3eceb-c294-4703-a15a-b1fe4de3376a-maas-api-tls\") pod \"maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw\" (UID: \"ebc3eceb-c294-4703-a15a-b1fe4de3376a\") " pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" Jul 07 07:32:20.476270 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.476236 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:20.523732 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.523703 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebc3eceb-c294-4703-a15a-b1fe4de3376a-openshift-service-ca\") pod \"maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw\" (UID: \"ebc3eceb-c294-4703-a15a-b1fe4de3376a\") " pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" Jul 07 07:32:20.523872 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.523738 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6z9g\" (UniqueName: \"kubernetes.io/projected/ebc3eceb-c294-4703-a15a-b1fe4de3376a-kube-api-access-g6z9g\") pod \"maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw\" (UID: \"ebc3eceb-c294-4703-a15a-b1fe4de3376a\") " pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" Jul 07 07:32:20.523872 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.523761 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ebc3eceb-c294-4703-a15a-b1fe4de3376a-maas-api-tls\") pod \"maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw\" (UID: \"ebc3eceb-c294-4703-a15a-b1fe4de3376a\") " pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" Jul 07 07:32:20.523945 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:32:20.523930 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-14bf96-serving-cert: secret "maas-api-e2e-api-a-14bf96-serving-cert" not found Jul 07 07:32:20.524034 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:32:20.524020 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebc3eceb-c294-4703-a15a-b1fe4de3376a-maas-api-tls podName:ebc3eceb-c294-4703-a15a-b1fe4de3376a nodeName:}" failed. No retries permitted until 2026-07-07 07:32:21.023997049 +0000 UTC m=+1996.359963347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ebc3eceb-c294-4703-a15a-b1fe4de3376a-maas-api-tls") pod "maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" (UID: "ebc3eceb-c294-4703-a15a-b1fe4de3376a") : secret "maas-api-e2e-api-a-14bf96-serving-cert" not found Jul 07 07:32:20.524456 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.524430 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebc3eceb-c294-4703-a15a-b1fe4de3376a-openshift-service-ca\") pod \"maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw\" (UID: \"ebc3eceb-c294-4703-a15a-b1fe4de3376a\") " pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" Jul 07 07:32:20.539239 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.539207 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6z9g\" (UniqueName: \"kubernetes.io/projected/ebc3eceb-c294-4703-a15a-b1fe4de3376a-kube-api-access-g6z9g\") pod \"maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw\" (UID: \"ebc3eceb-c294-4703-a15a-b1fe4de3376a\") " pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" Jul 07 07:32:20.588393 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:20.588286 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:21.027840 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:21.027809 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ebc3eceb-c294-4703-a15a-b1fe4de3376a-maas-api-tls\") pod \"maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw\" (UID: \"ebc3eceb-c294-4703-a15a-b1fe4de3376a\") " pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" Jul 07 07:32:21.030147 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:21.030117 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ebc3eceb-c294-4703-a15a-b1fe4de3376a-maas-api-tls\") pod \"maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw\" (UID: \"ebc3eceb-c294-4703-a15a-b1fe4de3376a\") " pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" Jul 07 07:32:21.158556 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:21.158516 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:21.257907 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:21.257865 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:21.312186 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:21.312101 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" Jul 07 07:32:21.449635 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:21.449610 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw"] Jul 07 07:32:21.451744 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:32:21.451718 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebc3eceb_c294_4703_a15a_b1fe4de3376a.slice/crio-986423c2d008f1e0cbb3b93d0abef89f55dc5911ffc6e56338d851f59b2c0a25 WatchSource:0}: Error finding container 986423c2d008f1e0cbb3b93d0abef89f55dc5911ffc6e56338d851f59b2c0a25: Status 404 returned error can't find the container with id 986423c2d008f1e0cbb3b93d0abef89f55dc5911ffc6e56338d851f59b2c0a25 Jul 07 07:32:22.101339 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:22.101302 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" event={"ID":"ebc3eceb-c294-4703-a15a-b1fe4de3376a","Type":"ContainerStarted","Data":"2403c164e36f577f9bb30f8042366ea3e5d6b856158b51d7e8d712223eba4187"} Jul 07 07:32:22.101514 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:22.101375 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" event={"ID":"ebc3eceb-c294-4703-a15a-b1fe4de3376a","Type":"ContainerStarted","Data":"986423c2d008f1e0cbb3b93d0abef89f55dc5911ffc6e56338d851f59b2c0a25"} Jul 07 07:32:22.101514 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:22.101448 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" Jul 07 07:32:22.129888 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:22.129845 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" podStartSLOduration=1.6862102970000001 podStartE2EDuration="2.129831927s" totalImagesPullingTime="443.62163ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:32:22.127044461 +0000 UTC m=+1997.463010796" watchObservedRunningTime="2026-07-07 07:32:22.129831927 +0000 UTC m=+1997.465798243" Jul 07 07:32:24.824822 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:24.824781 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:25.288977 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:25.288943 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:25.576213 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:25.576137 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:26.064853 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:26.064813 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:28.111444 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:28.111406 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" Jul 07 07:32:28.369257 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:28.369167 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:28.467986 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:28.467953 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:28.741763 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:28.741679 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv"] Jul 07 07:32:28.745276 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:28.745254 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" Jul 07 07:32:28.748046 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:28.748029 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-5b2516-serving-cert\"" Jul 07 07:32:28.765100 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:28.765077 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv"] Jul 07 07:32:28.792885 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:28.792863 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:28.798122 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:28.798097 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa1cfd88-29f7-4264-85ea-106d49c5d42d-maas-api-tls\") pod \"maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv\" (UID: \"fa1cfd88-29f7-4264-85ea-106d49c5d42d\") " pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" Jul 07 07:32:28.798205 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:28.798127 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt9w5\" (UniqueName: \"kubernetes.io/projected/fa1cfd88-29f7-4264-85ea-106d49c5d42d-kube-api-access-rt9w5\") pod \"maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv\" (UID: \"fa1cfd88-29f7-4264-85ea-106d49c5d42d\") " pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" Jul 07 07:32:28.798256 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:28.798216 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa1cfd88-29f7-4264-85ea-106d49c5d42d-openshift-service-ca\") pod \"maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv\" (UID: \"fa1cfd88-29f7-4264-85ea-106d49c5d42d\") " pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" Jul 07 07:32:28.899395 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:28.899355 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa1cfd88-29f7-4264-85ea-106d49c5d42d-maas-api-tls\") pod \"maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv\" (UID: \"fa1cfd88-29f7-4264-85ea-106d49c5d42d\") " pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" Jul 07 07:32:28.899557 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:28.899401 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt9w5\" (UniqueName: \"kubernetes.io/projected/fa1cfd88-29f7-4264-85ea-106d49c5d42d-kube-api-access-rt9w5\") pod \"maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv\" (UID: \"fa1cfd88-29f7-4264-85ea-106d49c5d42d\") " pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" Jul 07 07:32:28.899557 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:28.899467 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa1cfd88-29f7-4264-85ea-106d49c5d42d-openshift-service-ca\") pod \"maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv\" (UID: \"fa1cfd88-29f7-4264-85ea-106d49c5d42d\") " pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" Jul 07 07:32:28.899557 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:32:28.899492 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-5b2516-serving-cert: secret "maas-api-e2e-api-b-5b2516-serving-cert" not found Jul 07 07:32:28.899678 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:32:28.899560 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1cfd88-29f7-4264-85ea-106d49c5d42d-maas-api-tls podName:fa1cfd88-29f7-4264-85ea-106d49c5d42d nodeName:}" failed. No retries permitted until 2026-07-07 07:32:29.399545039 +0000 UTC m=+2004.735511339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fa1cfd88-29f7-4264-85ea-106d49c5d42d-maas-api-tls") pod "maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" (UID: "fa1cfd88-29f7-4264-85ea-106d49c5d42d") : secret "maas-api-e2e-api-b-5b2516-serving-cert" not found Jul 07 07:32:28.900091 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:28.900075 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa1cfd88-29f7-4264-85ea-106d49c5d42d-openshift-service-ca\") pod \"maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv\" (UID: \"fa1cfd88-29f7-4264-85ea-106d49c5d42d\") " pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" Jul 07 07:32:28.916535 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:28.916510 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt9w5\" (UniqueName: \"kubernetes.io/projected/fa1cfd88-29f7-4264-85ea-106d49c5d42d-kube-api-access-rt9w5\") pod \"maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv\" (UID: \"fa1cfd88-29f7-4264-85ea-106d49c5d42d\") " pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" Jul 07 07:32:29.263978 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:29.263948 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:29.365133 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:29.365095 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:29.404390 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:29.404354 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa1cfd88-29f7-4264-85ea-106d49c5d42d-maas-api-tls\") pod \"maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv\" (UID: \"fa1cfd88-29f7-4264-85ea-106d49c5d42d\") " pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" Jul 07 07:32:29.406759 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:29.406739 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa1cfd88-29f7-4264-85ea-106d49c5d42d-maas-api-tls\") pod \"maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv\" (UID: \"fa1cfd88-29f7-4264-85ea-106d49c5d42d\") " pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" Jul 07 07:32:29.655666 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:29.655575 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" Jul 07 07:32:29.793022 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:29.792995 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv"] Jul 07 07:32:29.795086 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:32:29.795058 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa1cfd88_29f7_4264_85ea_106d49c5d42d.slice/crio-06d5851d0b01cb7b7e3bebffa15f35a8f21ad4ef81cbb4e0da5e64b2ac83fe8b WatchSource:0}: Error finding container 06d5851d0b01cb7b7e3bebffa15f35a8f21ad4ef81cbb4e0da5e64b2ac83fe8b: Status 404 returned error can't find the container with id 06d5851d0b01cb7b7e3bebffa15f35a8f21ad4ef81cbb4e0da5e64b2ac83fe8b Jul 07 07:32:30.140764 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:30.140728 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" event={"ID":"fa1cfd88-29f7-4264-85ea-106d49c5d42d","Type":"ContainerStarted","Data":"06d5851d0b01cb7b7e3bebffa15f35a8f21ad4ef81cbb4e0da5e64b2ac83fe8b"} Jul 07 07:32:31.146536 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:31.146504 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" event={"ID":"fa1cfd88-29f7-4264-85ea-106d49c5d42d","Type":"ContainerStarted","Data":"a40739bf1456beef7fd467756ed58e08590dde2574430cefbb6daf73ca2c8e4e"} Jul 07 07:32:31.146895 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:31.146640 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" Jul 07 07:32:31.167159 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:31.167116 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" podStartSLOduration=2.70849552 podStartE2EDuration="3.167103101s" totalImagesPullingTime="458.607581ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:32:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:32:31.164927236 +0000 UTC m=+2006.500893570" watchObservedRunningTime="2026-07-07 07:32:31.167103101 +0000 UTC m=+2006.503069440" Jul 07 07:32:32.877567 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:32.877534 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:37.156693 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:37.156666 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" Jul 07 07:32:38.749962 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:38.749929 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:38.821193 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:38.821155 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:41.002425 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:41.002390 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:41.299165 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:41.299074 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:41.398203 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:41.398170 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:41.558717 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:41.558647 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw"] Jul 07 07:32:41.558886 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:41.558862 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" podUID="ebc3eceb-c294-4703-a15a-b1fe4de3376a" containerName="maas-api" containerID="cri-o://2403c164e36f577f9bb30f8042366ea3e5d6b856158b51d7e8d712223eba4187" gracePeriod=30 Jul 07 07:32:41.585008 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:41.584982 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:41.811346 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:41.811269 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" Jul 07 07:32:41.914494 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:41.914464 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebc3eceb-c294-4703-a15a-b1fe4de3376a-openshift-service-ca\") pod \"ebc3eceb-c294-4703-a15a-b1fe4de3376a\" (UID: \"ebc3eceb-c294-4703-a15a-b1fe4de3376a\") " Jul 07 07:32:41.914662 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:41.914505 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ebc3eceb-c294-4703-a15a-b1fe4de3376a-maas-api-tls\") pod \"ebc3eceb-c294-4703-a15a-b1fe4de3376a\" (UID: \"ebc3eceb-c294-4703-a15a-b1fe4de3376a\") " Jul 07 07:32:41.914662 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:41.914558 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6z9g\" (UniqueName: \"kubernetes.io/projected/ebc3eceb-c294-4703-a15a-b1fe4de3376a-kube-api-access-g6z9g\") pod \"ebc3eceb-c294-4703-a15a-b1fe4de3376a\" (UID: \"ebc3eceb-c294-4703-a15a-b1fe4de3376a\") " Jul 07 07:32:41.914879 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:41.914856 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebc3eceb-c294-4703-a15a-b1fe4de3376a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ebc3eceb-c294-4703-a15a-b1fe4de3376a" (UID: "ebc3eceb-c294-4703-a15a-b1fe4de3376a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:32:41.916625 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:41.916602 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebc3eceb-c294-4703-a15a-b1fe4de3376a-kube-api-access-g6z9g" (OuterVolumeSpecName: "kube-api-access-g6z9g") pod "ebc3eceb-c294-4703-a15a-b1fe4de3376a" (UID: "ebc3eceb-c294-4703-a15a-b1fe4de3376a"). InnerVolumeSpecName "kube-api-access-g6z9g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:32:41.916704 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:41.916674 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebc3eceb-c294-4703-a15a-b1fe4de3376a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ebc3eceb-c294-4703-a15a-b1fe4de3376a" (UID: "ebc3eceb-c294-4703-a15a-b1fe4de3376a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:32:42.015884 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:42.015854 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebc3eceb-c294-4703-a15a-b1fe4de3376a-openshift-service-ca\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:32:42.015884 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:42.015879 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ebc3eceb-c294-4703-a15a-b1fe4de3376a-maas-api-tls\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:32:42.015884 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:42.015888 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6z9g\" (UniqueName: \"kubernetes.io/projected/ebc3eceb-c294-4703-a15a-b1fe4de3376a-kube-api-access-g6z9g\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:32:42.196573 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:42.196493 2573 generic.go:356] "Generic (PLEG): container finished" podID="ebc3eceb-c294-4703-a15a-b1fe4de3376a" containerID="2403c164e36f577f9bb30f8042366ea3e5d6b856158b51d7e8d712223eba4187" exitCode=0 Jul 07 07:32:42.196573 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:42.196534 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" event={"ID":"ebc3eceb-c294-4703-a15a-b1fe4de3376a","Type":"ContainerDied","Data":"2403c164e36f577f9bb30f8042366ea3e5d6b856158b51d7e8d712223eba4187"} Jul 07 07:32:42.196573 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:42.196556 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" Jul 07 07:32:42.196814 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:42.196598 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw" event={"ID":"ebc3eceb-c294-4703-a15a-b1fe4de3376a","Type":"ContainerDied","Data":"986423c2d008f1e0cbb3b93d0abef89f55dc5911ffc6e56338d851f59b2c0a25"} Jul 07 07:32:42.196814 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:42.196632 2573 scope.go:117] "RemoveContainer" containerID="2403c164e36f577f9bb30f8042366ea3e5d6b856158b51d7e8d712223eba4187" Jul 07 07:32:42.205908 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:42.205890 2573 scope.go:117] "RemoveContainer" containerID="2403c164e36f577f9bb30f8042366ea3e5d6b856158b51d7e8d712223eba4187" Jul 07 07:32:42.206178 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:32:42.206161 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2403c164e36f577f9bb30f8042366ea3e5d6b856158b51d7e8d712223eba4187\": container with ID starting with 2403c164e36f577f9bb30f8042366ea3e5d6b856158b51d7e8d712223eba4187 not found: ID does not exist" containerID="2403c164e36f577f9bb30f8042366ea3e5d6b856158b51d7e8d712223eba4187" Jul 07 07:32:42.206225 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:42.206187 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2403c164e36f577f9bb30f8042366ea3e5d6b856158b51d7e8d712223eba4187"} err="failed to get container status \"2403c164e36f577f9bb30f8042366ea3e5d6b856158b51d7e8d712223eba4187\": rpc error: code = NotFound desc = could not find container \"2403c164e36f577f9bb30f8042366ea3e5d6b856158b51d7e8d712223eba4187\": container with ID starting with 2403c164e36f577f9bb30f8042366ea3e5d6b856158b51d7e8d712223eba4187 not found: ID does not exist" Jul 07 07:32:42.221287 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:42.221262 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw"] Jul 07 07:32:42.233878 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:42.233856 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-14bf96-fb5c5d77d-whvfw"] Jul 07 07:32:43.108643 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:43.108615 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebc3eceb-c294-4703-a15a-b1fe4de3376a" path="/var/lib/kubelet/pods/ebc3eceb-c294-4703-a15a-b1fe4de3376a/volumes" Jul 07 07:32:47.631270 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:47.631239 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:47.733429 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:47.733392 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:47.885796 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:47.885713 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:48.100320 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:48.100289 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:48.314436 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:48.314379 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:48.725223 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:48.725133 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:48.854057 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:48.854029 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:48.955149 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:48.955118 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:49.193032 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:49.193000 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv"] Jul 07 07:32:49.193215 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:49.193190 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" podUID="fa1cfd88-29f7-4264-85ea-106d49c5d42d" containerName="maas-api" containerID="cri-o://a40739bf1456beef7fd467756ed58e08590dde2574430cefbb6daf73ca2c8e4e" gracePeriod=30 Jul 07 07:32:49.230862 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:49.230837 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:49.440036 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:49.440013 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" Jul 07 07:32:49.581108 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:49.581079 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa1cfd88-29f7-4264-85ea-106d49c5d42d-openshift-service-ca\") pod \"fa1cfd88-29f7-4264-85ea-106d49c5d42d\" (UID: \"fa1cfd88-29f7-4264-85ea-106d49c5d42d\") " Jul 07 07:32:49.581285 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:49.581183 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rt9w5\" (UniqueName: \"kubernetes.io/projected/fa1cfd88-29f7-4264-85ea-106d49c5d42d-kube-api-access-rt9w5\") pod \"fa1cfd88-29f7-4264-85ea-106d49c5d42d\" (UID: \"fa1cfd88-29f7-4264-85ea-106d49c5d42d\") " Jul 07 07:32:49.581285 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:49.581260 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa1cfd88-29f7-4264-85ea-106d49c5d42d-maas-api-tls\") pod \"fa1cfd88-29f7-4264-85ea-106d49c5d42d\" (UID: \"fa1cfd88-29f7-4264-85ea-106d49c5d42d\") " Jul 07 07:32:49.581537 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:49.581512 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa1cfd88-29f7-4264-85ea-106d49c5d42d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fa1cfd88-29f7-4264-85ea-106d49c5d42d" (UID: "fa1cfd88-29f7-4264-85ea-106d49c5d42d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:32:49.583264 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:49.583236 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa1cfd88-29f7-4264-85ea-106d49c5d42d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fa1cfd88-29f7-4264-85ea-106d49c5d42d" (UID: "fa1cfd88-29f7-4264-85ea-106d49c5d42d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:32:49.583472 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:49.583311 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa1cfd88-29f7-4264-85ea-106d49c5d42d-kube-api-access-rt9w5" (OuterVolumeSpecName: "kube-api-access-rt9w5") pod "fa1cfd88-29f7-4264-85ea-106d49c5d42d" (UID: "fa1cfd88-29f7-4264-85ea-106d49c5d42d"). InnerVolumeSpecName "kube-api-access-rt9w5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:32:49.682338 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:49.682296 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt9w5\" (UniqueName: \"kubernetes.io/projected/fa1cfd88-29f7-4264-85ea-106d49c5d42d-kube-api-access-rt9w5\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:32:49.682461 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:49.682344 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa1cfd88-29f7-4264-85ea-106d49c5d42d-maas-api-tls\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:32:49.682461 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:49.682358 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa1cfd88-29f7-4264-85ea-106d49c5d42d-openshift-service-ca\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:32:50.235169 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:50.235137 2573 generic.go:356] "Generic (PLEG): container finished" podID="fa1cfd88-29f7-4264-85ea-106d49c5d42d" containerID="a40739bf1456beef7fd467756ed58e08590dde2574430cefbb6daf73ca2c8e4e" exitCode=0 Jul 07 07:32:50.235605 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:50.235194 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" Jul 07 07:32:50.235605 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:50.235205 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" event={"ID":"fa1cfd88-29f7-4264-85ea-106d49c5d42d","Type":"ContainerDied","Data":"a40739bf1456beef7fd467756ed58e08590dde2574430cefbb6daf73ca2c8e4e"} Jul 07 07:32:50.235605 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:50.235266 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv" event={"ID":"fa1cfd88-29f7-4264-85ea-106d49c5d42d","Type":"ContainerDied","Data":"06d5851d0b01cb7b7e3bebffa15f35a8f21ad4ef81cbb4e0da5e64b2ac83fe8b"} Jul 07 07:32:50.235605 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:50.235297 2573 scope.go:117] "RemoveContainer" containerID="a40739bf1456beef7fd467756ed58e08590dde2574430cefbb6daf73ca2c8e4e" Jul 07 07:32:50.243908 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:50.243890 2573 scope.go:117] "RemoveContainer" containerID="a40739bf1456beef7fd467756ed58e08590dde2574430cefbb6daf73ca2c8e4e" Jul 07 07:32:50.244262 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:32:50.244237 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a40739bf1456beef7fd467756ed58e08590dde2574430cefbb6daf73ca2c8e4e\": container with ID starting with a40739bf1456beef7fd467756ed58e08590dde2574430cefbb6daf73ca2c8e4e not found: ID does not exist" containerID="a40739bf1456beef7fd467756ed58e08590dde2574430cefbb6daf73ca2c8e4e" Jul 07 07:32:50.244345 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:50.244272 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a40739bf1456beef7fd467756ed58e08590dde2574430cefbb6daf73ca2c8e4e"} err="failed to get container status \"a40739bf1456beef7fd467756ed58e08590dde2574430cefbb6daf73ca2c8e4e\": rpc error: code = NotFound desc = could not find container \"a40739bf1456beef7fd467756ed58e08590dde2574430cefbb6daf73ca2c8e4e\": container with ID starting with a40739bf1456beef7fd467756ed58e08590dde2574430cefbb6daf73ca2c8e4e not found: ID does not exist" Jul 07 07:32:50.260421 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:50.260395 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv"] Jul 07 07:32:50.263119 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:50.263099 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-5b2516-dbbfb9c66-9h8qv"] Jul 07 07:32:51.108868 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:51.108826 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa1cfd88-29f7-4264-85ea-106d49c5d42d" path="/var/lib/kubelet/pods/fa1cfd88-29f7-4264-85ea-106d49c5d42d/volumes" Jul 07 07:32:54.870683 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:54.870650 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:54.971336 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:54.971295 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:55.124122 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:55.124048 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:55.344856 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:55.344827 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:55.667456 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:55.667420 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:55.980827 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:55.980737 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:56.272783 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:56.272747 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:58.762149 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.762117 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx"] Jul 07 07:32:58.762723 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.762706 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebc3eceb-c294-4703-a15a-b1fe4de3376a" containerName="maas-api" Jul 07 07:32:58.762785 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.762726 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebc3eceb-c294-4703-a15a-b1fe4de3376a" containerName="maas-api" Jul 07 07:32:58.762785 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.762744 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa1cfd88-29f7-4264-85ea-106d49c5d42d" containerName="maas-api" Jul 07 07:32:58.762785 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.762752 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa1cfd88-29f7-4264-85ea-106d49c5d42d" containerName="maas-api" Jul 07 07:32:58.762933 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.762850 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebc3eceb-c294-4703-a15a-b1fe4de3376a" containerName="maas-api" Jul 07 07:32:58.762933 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.762863 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa1cfd88-29f7-4264-85ea-106d49c5d42d" containerName="maas-api" Jul 07 07:32:58.767184 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.767161 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" Jul 07 07:32:58.769762 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.769737 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-083d4e-serving-cert\"" Jul 07 07:32:58.779065 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.779039 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx"] Jul 07 07:32:58.860606 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.860573 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:58.866830 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.866804 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/849545ba-ee76-4812-b2cc-bbe1f2337ffc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx\" (UID: \"849545ba-ee76-4812-b2cc-bbe1f2337ffc\") " pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" Jul 07 07:32:58.866976 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.866846 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdnmn\" (UniqueName: \"kubernetes.io/projected/849545ba-ee76-4812-b2cc-bbe1f2337ffc-kube-api-access-jdnmn\") pod \"maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx\" (UID: \"849545ba-ee76-4812-b2cc-bbe1f2337ffc\") " pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" Jul 07 07:32:58.866976 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.866910 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/849545ba-ee76-4812-b2cc-bbe1f2337ffc-maas-api-tls\") pod \"maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx\" (UID: \"849545ba-ee76-4812-b2cc-bbe1f2337ffc\") " pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" Jul 07 07:32:58.961335 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.961291 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:58.967305 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.967278 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/849545ba-ee76-4812-b2cc-bbe1f2337ffc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx\" (UID: \"849545ba-ee76-4812-b2cc-bbe1f2337ffc\") " pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" Jul 07 07:32:58.967469 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.967339 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdnmn\" (UniqueName: \"kubernetes.io/projected/849545ba-ee76-4812-b2cc-bbe1f2337ffc-kube-api-access-jdnmn\") pod \"maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx\" (UID: \"849545ba-ee76-4812-b2cc-bbe1f2337ffc\") " pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" Jul 07 07:32:58.967469 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.967395 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/849545ba-ee76-4812-b2cc-bbe1f2337ffc-maas-api-tls\") pod \"maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx\" (UID: \"849545ba-ee76-4812-b2cc-bbe1f2337ffc\") " pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" Jul 07 07:32:58.967584 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:32:58.967552 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-083d4e-serving-cert: secret "maas-api-e2e-shared-a-083d4e-serving-cert" not found Jul 07 07:32:58.967638 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:32:58.967623 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/849545ba-ee76-4812-b2cc-bbe1f2337ffc-maas-api-tls podName:849545ba-ee76-4812-b2cc-bbe1f2337ffc nodeName:}" failed. No retries permitted until 2026-07-07 07:32:59.467602647 +0000 UTC m=+2034.803568945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/849545ba-ee76-4812-b2cc-bbe1f2337ffc-maas-api-tls") pod "maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" (UID: "849545ba-ee76-4812-b2cc-bbe1f2337ffc") : secret "maas-api-e2e-shared-a-083d4e-serving-cert" not found Jul 07 07:32:58.968020 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.968003 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/849545ba-ee76-4812-b2cc-bbe1f2337ffc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx\" (UID: \"849545ba-ee76-4812-b2cc-bbe1f2337ffc\") " pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" Jul 07 07:32:58.978311 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:58.978280 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdnmn\" (UniqueName: \"kubernetes.io/projected/849545ba-ee76-4812-b2cc-bbe1f2337ffc-kube-api-access-jdnmn\") pod \"maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx\" (UID: \"849545ba-ee76-4812-b2cc-bbe1f2337ffc\") " pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" Jul 07 07:32:59.208459 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:59.208384 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:59.389982 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:59.389950 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:59.472032 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:59.471955 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/849545ba-ee76-4812-b2cc-bbe1f2337ffc-maas-api-tls\") pod \"maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx\" (UID: \"849545ba-ee76-4812-b2cc-bbe1f2337ffc\") " pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" Jul 07 07:32:59.474313 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:59.474285 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/849545ba-ee76-4812-b2cc-bbe1f2337ffc-maas-api-tls\") pod \"maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx\" (UID: \"849545ba-ee76-4812-b2cc-bbe1f2337ffc\") " pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" Jul 07 07:32:59.613261 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:59.613227 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:32:59.678666 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:59.678635 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" Jul 07 07:32:59.823734 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:32:59.823708 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx"] Jul 07 07:32:59.825530 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:32:59.825501 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod849545ba_ee76_4812_b2cc_bbe1f2337ffc.slice/crio-e72e64b8f3af0e7f48c421e6ab75e92971e4dd80673b3f74ed4df7d5aa949b60 WatchSource:0}: Error finding container e72e64b8f3af0e7f48c421e6ab75e92971e4dd80673b3f74ed4df7d5aa949b60: Status 404 returned error can't find the container with id e72e64b8f3af0e7f48c421e6ab75e92971e4dd80673b3f74ed4df7d5aa949b60 Jul 07 07:33:00.285386 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:00.285352 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" event={"ID":"849545ba-ee76-4812-b2cc-bbe1f2337ffc","Type":"ContainerStarted","Data":"e72e64b8f3af0e7f48c421e6ab75e92971e4dd80673b3f74ed4df7d5aa949b60"} Jul 07 07:33:01.291380 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:01.291346 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" event={"ID":"849545ba-ee76-4812-b2cc-bbe1f2337ffc","Type":"ContainerStarted","Data":"0c3d440ea02de33af559cd7e564d411d1ab0b5b6dca338ceb1e5cd8ac8a587a2"} Jul 07 07:33:01.291767 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:01.291501 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" Jul 07 07:33:01.313429 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:01.313383 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" podStartSLOduration=2.814665036 podStartE2EDuration="3.313368298s" totalImagesPullingTime="498.703262ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:33:01.311245497 +0000 UTC m=+2036.647211814" watchObservedRunningTime="2026-07-07 07:33:01.313368298 +0000 UTC m=+2036.649334634" Jul 07 07:33:03.055092 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:03.055062 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:03.383020 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:03.382942 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:03.678167 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:03.678079 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:03.914385 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:03.914355 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:06.602361 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:06.602296 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:06.669865 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:06.669835 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b"] Jul 07 07:33:06.674782 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:06.674759 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" Jul 07 07:33:06.677388 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:06.677367 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-1082ad-serving-cert\"" Jul 07 07:33:06.686894 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:06.686864 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b"] Jul 07 07:33:06.692410 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:06.692389 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:06.727976 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:06.727953 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw2dm\" (UniqueName: \"kubernetes.io/projected/dc357ad5-5faf-4af3-8744-91fdca5729c9-kube-api-access-kw2dm\") pod \"maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b\" (UID: \"dc357ad5-5faf-4af3-8744-91fdca5729c9\") " pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" Jul 07 07:33:06.728099 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:06.728012 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc357ad5-5faf-4af3-8744-91fdca5729c9-maas-api-tls\") pod \"maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b\" (UID: \"dc357ad5-5faf-4af3-8744-91fdca5729c9\") " pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" Jul 07 07:33:06.728099 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:06.728088 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc357ad5-5faf-4af3-8744-91fdca5729c9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b\" (UID: \"dc357ad5-5faf-4af3-8744-91fdca5729c9\") " pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" Jul 07 07:33:06.777239 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:06.777212 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:06.829141 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:06.829120 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc357ad5-5faf-4af3-8744-91fdca5729c9-maas-api-tls\") pod \"maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b\" (UID: \"dc357ad5-5faf-4af3-8744-91fdca5729c9\") " pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" Jul 07 07:33:06.829248 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:06.829160 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc357ad5-5faf-4af3-8744-91fdca5729c9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b\" (UID: \"dc357ad5-5faf-4af3-8744-91fdca5729c9\") " pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" Jul 07 07:33:06.829248 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:06.829199 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw2dm\" (UniqueName: \"kubernetes.io/projected/dc357ad5-5faf-4af3-8744-91fdca5729c9-kube-api-access-kw2dm\") pod \"maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b\" (UID: \"dc357ad5-5faf-4af3-8744-91fdca5729c9\") " pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" Jul 07 07:33:06.829366 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:33:06.829266 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-1082ad-serving-cert: secret "maas-api-e2e-shared-b-1082ad-serving-cert" not found Jul 07 07:33:06.829419 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:33:06.829365 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc357ad5-5faf-4af3-8744-91fdca5729c9-maas-api-tls podName:dc357ad5-5faf-4af3-8744-91fdca5729c9 nodeName:}" failed. No retries permitted until 2026-07-07 07:33:07.329345405 +0000 UTC m=+2042.665311715 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dc357ad5-5faf-4af3-8744-91fdca5729c9-maas-api-tls") pod "maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" (UID: "dc357ad5-5faf-4af3-8744-91fdca5729c9") : secret "maas-api-e2e-shared-b-1082ad-serving-cert" not found Jul 07 07:33:06.829845 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:06.829827 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc357ad5-5faf-4af3-8744-91fdca5729c9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b\" (UID: \"dc357ad5-5faf-4af3-8744-91fdca5729c9\") " pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" Jul 07 07:33:06.841546 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:06.841524 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw2dm\" (UniqueName: \"kubernetes.io/projected/dc357ad5-5faf-4af3-8744-91fdca5729c9-kube-api-access-kw2dm\") pod \"maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b\" (UID: \"dc357ad5-5faf-4af3-8744-91fdca5729c9\") " pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" Jul 07 07:33:07.173535 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:07.173502 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:07.273878 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:07.273847 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:07.301467 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:07.301445 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" Jul 07 07:33:07.333935 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:07.333905 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc357ad5-5faf-4af3-8744-91fdca5729c9-maas-api-tls\") pod \"maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b\" (UID: \"dc357ad5-5faf-4af3-8744-91fdca5729c9\") " pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" Jul 07 07:33:07.338143 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:07.338115 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc357ad5-5faf-4af3-8744-91fdca5729c9-maas-api-tls\") pod \"maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b\" (UID: \"dc357ad5-5faf-4af3-8744-91fdca5729c9\") " pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" Jul 07 07:33:07.375061 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:07.375034 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:07.585706 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:07.585679 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" Jul 07 07:33:07.722534 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:07.722511 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b"] Jul 07 07:33:07.724425 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:33:07.724395 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc357ad5_5faf_4af3_8744_91fdca5729c9.slice/crio-3343b087011fe49810b7d57c611d3a9c76b2aacc48fe6d8f7fb6a727a0cca5df WatchSource:0}: Error finding container 3343b087011fe49810b7d57c611d3a9c76b2aacc48fe6d8f7fb6a727a0cca5df: Status 404 returned error can't find the container with id 3343b087011fe49810b7d57c611d3a9c76b2aacc48fe6d8f7fb6a727a0cca5df Jul 07 07:33:08.325012 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:08.324932 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" event={"ID":"dc357ad5-5faf-4af3-8744-91fdca5729c9","Type":"ContainerStarted","Data":"704a58da1fbd74633b11b096eca67f98519fe581993f5a7c87463d9439da7e74"} Jul 07 07:33:08.325012 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:08.324982 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" event={"ID":"dc357ad5-5faf-4af3-8744-91fdca5729c9","Type":"ContainerStarted","Data":"3343b087011fe49810b7d57c611d3a9c76b2aacc48fe6d8f7fb6a727a0cca5df"} Jul 07 07:33:08.325284 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:08.325031 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" Jul 07 07:33:08.344745 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:08.344702 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" podStartSLOduration=1.997528007 podStartE2EDuration="2.34468788s" totalImagesPullingTime="347.159873ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:33:08.342299337 +0000 UTC m=+2043.678265654" watchObservedRunningTime="2026-07-07 07:33:08.34468788 +0000 UTC m=+2043.680654218" Jul 07 07:33:11.103391 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:11.103354 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:14.334805 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:14.334774 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" Jul 07 07:33:14.586129 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:14.586049 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:20.365531 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:20.365501 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:20.464751 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:20.464713 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:20.638515 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:20.638391 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:28.082363 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:28.082246 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:28.185375 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:28.185305 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:33:43.342519 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:33:43.342480 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:34:05.190496 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:34:05.190468 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:34:05.198472 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:34:05.198453 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:34:51.564799 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:34:51.564764 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:35:35.955343 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:35:35.955297 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:35:36.900368 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:35:36.900306 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:35:53.345047 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:35:53.345018 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:36:16.235603 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:36:16.235563 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:36:57.532627 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:36:57.532589 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:36:58.514823 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:36:58.514782 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:37:05.932154 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:37:05.932115 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:37:23.542671 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:37:23.542638 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:37:35.261920 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:37:35.261884 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:37:42.201240 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:37:42.201207 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:37:42.302130 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:37:42.302097 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:38:21.276097 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:21.276065 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:38:21.366960 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:21.366933 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:38:22.785468 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:22.785428 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:38:23.661537 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:23.661503 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:38:24.862266 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:24.862230 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:38:25.295496 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.295462 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:38:25.455076 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.455044 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx"] Jul 07 07:38:25.455264 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.455243 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" podUID="849545ba-ee76-4812-b2cc-bbe1f2337ffc" containerName="maas-api" containerID="cri-o://0c3d440ea02de33af559cd7e564d411d1ab0b5b6dca338ceb1e5cd8ac8a587a2" gracePeriod=30 Jul 07 07:38:25.525134 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.525106 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:38:25.699391 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.699370 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" Jul 07 07:38:25.820587 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.820556 2573 generic.go:356] "Generic (PLEG): container finished" podID="849545ba-ee76-4812-b2cc-bbe1f2337ffc" containerID="0c3d440ea02de33af559cd7e564d411d1ab0b5b6dca338ceb1e5cd8ac8a587a2" exitCode=0 Jul 07 07:38:25.820766 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.820595 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" Jul 07 07:38:25.820766 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.820655 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" event={"ID":"849545ba-ee76-4812-b2cc-bbe1f2337ffc","Type":"ContainerDied","Data":"0c3d440ea02de33af559cd7e564d411d1ab0b5b6dca338ceb1e5cd8ac8a587a2"} Jul 07 07:38:25.820766 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.820702 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx" event={"ID":"849545ba-ee76-4812-b2cc-bbe1f2337ffc","Type":"ContainerDied","Data":"e72e64b8f3af0e7f48c421e6ab75e92971e4dd80673b3f74ed4df7d5aa949b60"} Jul 07 07:38:25.820766 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.820729 2573 scope.go:117] "RemoveContainer" containerID="0c3d440ea02de33af559cd7e564d411d1ab0b5b6dca338ceb1e5cd8ac8a587a2" Jul 07 07:38:25.829100 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.829083 2573 scope.go:117] "RemoveContainer" containerID="0c3d440ea02de33af559cd7e564d411d1ab0b5b6dca338ceb1e5cd8ac8a587a2" Jul 07 07:38:25.829368 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:38:25.829349 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c3d440ea02de33af559cd7e564d411d1ab0b5b6dca338ceb1e5cd8ac8a587a2\": container with ID starting with 0c3d440ea02de33af559cd7e564d411d1ab0b5b6dca338ceb1e5cd8ac8a587a2 not found: ID does not exist" containerID="0c3d440ea02de33af559cd7e564d411d1ab0b5b6dca338ceb1e5cd8ac8a587a2" Jul 07 07:38:25.829434 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.829385 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3d440ea02de33af559cd7e564d411d1ab0b5b6dca338ceb1e5cd8ac8a587a2"} err="failed to get container status \"0c3d440ea02de33af559cd7e564d411d1ab0b5b6dca338ceb1e5cd8ac8a587a2\": rpc error: code = NotFound desc = could not find container \"0c3d440ea02de33af559cd7e564d411d1ab0b5b6dca338ceb1e5cd8ac8a587a2\": container with ID starting with 0c3d440ea02de33af559cd7e564d411d1ab0b5b6dca338ceb1e5cd8ac8a587a2 not found: ID does not exist" Jul 07 07:38:25.831636 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.831621 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdnmn\" (UniqueName: \"kubernetes.io/projected/849545ba-ee76-4812-b2cc-bbe1f2337ffc-kube-api-access-jdnmn\") pod \"849545ba-ee76-4812-b2cc-bbe1f2337ffc\" (UID: \"849545ba-ee76-4812-b2cc-bbe1f2337ffc\") " Jul 07 07:38:25.831730 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.831719 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/849545ba-ee76-4812-b2cc-bbe1f2337ffc-openshift-service-ca\") pod \"849545ba-ee76-4812-b2cc-bbe1f2337ffc\" (UID: \"849545ba-ee76-4812-b2cc-bbe1f2337ffc\") " Jul 07 07:38:25.831772 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.831743 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/849545ba-ee76-4812-b2cc-bbe1f2337ffc-maas-api-tls\") pod \"849545ba-ee76-4812-b2cc-bbe1f2337ffc\" (UID: \"849545ba-ee76-4812-b2cc-bbe1f2337ffc\") " Jul 07 07:38:25.832033 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.832013 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/849545ba-ee76-4812-b2cc-bbe1f2337ffc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "849545ba-ee76-4812-b2cc-bbe1f2337ffc" (UID: "849545ba-ee76-4812-b2cc-bbe1f2337ffc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:38:25.833760 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.833736 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/849545ba-ee76-4812-b2cc-bbe1f2337ffc-kube-api-access-jdnmn" (OuterVolumeSpecName: "kube-api-access-jdnmn") pod "849545ba-ee76-4812-b2cc-bbe1f2337ffc" (UID: "849545ba-ee76-4812-b2cc-bbe1f2337ffc"). InnerVolumeSpecName "kube-api-access-jdnmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:38:25.833760 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.833750 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/849545ba-ee76-4812-b2cc-bbe1f2337ffc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "849545ba-ee76-4812-b2cc-bbe1f2337ffc" (UID: "849545ba-ee76-4812-b2cc-bbe1f2337ffc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:38:25.933166 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.933131 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdnmn\" (UniqueName: \"kubernetes.io/projected/849545ba-ee76-4812-b2cc-bbe1f2337ffc-kube-api-access-jdnmn\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:38:25.933166 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.933160 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/849545ba-ee76-4812-b2cc-bbe1f2337ffc-openshift-service-ca\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:38:25.933166 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:25.933170 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/849545ba-ee76-4812-b2cc-bbe1f2337ffc-maas-api-tls\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:38:26.153275 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:26.153241 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx"] Jul 07 07:38:26.155026 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:26.155006 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-083d4e-785867cb67-bv2gx"] Jul 07 07:38:27.108479 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:27.108437 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="849545ba-ee76-4812-b2cc-bbe1f2337ffc" path="/var/lib/kubelet/pods/849545ba-ee76-4812-b2cc-bbe1f2337ffc/volumes" Jul 07 07:38:28.998494 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:28.998458 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:38:29.100984 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:29.100946 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:38:30.362570 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:30.362541 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:38:55.054211 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:38:55.054147 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:39:05.217210 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:05.217176 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:39:05.232234 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:05.232212 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:39:08.741556 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:08.741522 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:39:09.117255 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:09.117223 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:39:09.317468 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:09.317435 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:39:09.581782 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:09.581748 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:39:10.122059 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:10.122031 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:39:10.342127 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:10.342096 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b"] Jul 07 07:39:10.342371 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:10.342307 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" podUID="dc357ad5-5faf-4af3-8744-91fdca5729c9" containerName="maas-api" containerID="cri-o://704a58da1fbd74633b11b096eca67f98519fe581993f5a7c87463d9439da7e74" gracePeriod=30 Jul 07 07:39:10.414921 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:10.414850 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:39:10.591964 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:10.591940 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" Jul 07 07:39:10.718807 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:10.718731 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc357ad5-5faf-4af3-8744-91fdca5729c9-openshift-service-ca\") pod \"dc357ad5-5faf-4af3-8744-91fdca5729c9\" (UID: \"dc357ad5-5faf-4af3-8744-91fdca5729c9\") " Jul 07 07:39:10.718807 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:10.718805 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kw2dm\" (UniqueName: \"kubernetes.io/projected/dc357ad5-5faf-4af3-8744-91fdca5729c9-kube-api-access-kw2dm\") pod \"dc357ad5-5faf-4af3-8744-91fdca5729c9\" (UID: \"dc357ad5-5faf-4af3-8744-91fdca5729c9\") " Jul 07 07:39:10.719086 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:10.718847 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc357ad5-5faf-4af3-8744-91fdca5729c9-maas-api-tls\") pod \"dc357ad5-5faf-4af3-8744-91fdca5729c9\" (UID: \"dc357ad5-5faf-4af3-8744-91fdca5729c9\") " Jul 07 07:39:10.719167 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:10.719137 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc357ad5-5faf-4af3-8744-91fdca5729c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dc357ad5-5faf-4af3-8744-91fdca5729c9" (UID: "dc357ad5-5faf-4af3-8744-91fdca5729c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:39:10.720890 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:10.720865 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc357ad5-5faf-4af3-8744-91fdca5729c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dc357ad5-5faf-4af3-8744-91fdca5729c9" (UID: "dc357ad5-5faf-4af3-8744-91fdca5729c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:39:10.720890 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:10.720883 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc357ad5-5faf-4af3-8744-91fdca5729c9-kube-api-access-kw2dm" (OuterVolumeSpecName: "kube-api-access-kw2dm") pod "dc357ad5-5faf-4af3-8744-91fdca5729c9" (UID: "dc357ad5-5faf-4af3-8744-91fdca5729c9"). InnerVolumeSpecName "kube-api-access-kw2dm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:39:10.819405 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:10.819374 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc357ad5-5faf-4af3-8744-91fdca5729c9-maas-api-tls\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:39:10.819405 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:10.819398 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc357ad5-5faf-4af3-8744-91fdca5729c9-openshift-service-ca\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:39:10.819405 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:10.819407 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kw2dm\" (UniqueName: \"kubernetes.io/projected/dc357ad5-5faf-4af3-8744-91fdca5729c9-kube-api-access-kw2dm\") on node \"ip-10-0-131-241.ec2.internal\" DevicePath \"\"" Jul 07 07:39:11.030232 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:11.030198 2573 generic.go:356] "Generic (PLEG): container finished" podID="dc357ad5-5faf-4af3-8744-91fdca5729c9" containerID="704a58da1fbd74633b11b096eca67f98519fe581993f5a7c87463d9439da7e74" exitCode=0 Jul 07 07:39:11.030418 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:11.030241 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" Jul 07 07:39:11.030418 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:11.030273 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" event={"ID":"dc357ad5-5faf-4af3-8744-91fdca5729c9","Type":"ContainerDied","Data":"704a58da1fbd74633b11b096eca67f98519fe581993f5a7c87463d9439da7e74"} Jul 07 07:39:11.030418 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:11.030367 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b" event={"ID":"dc357ad5-5faf-4af3-8744-91fdca5729c9","Type":"ContainerDied","Data":"3343b087011fe49810b7d57c611d3a9c76b2aacc48fe6d8f7fb6a727a0cca5df"} Jul 07 07:39:11.030418 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:11.030391 2573 scope.go:117] "RemoveContainer" containerID="704a58da1fbd74633b11b096eca67f98519fe581993f5a7c87463d9439da7e74" Jul 07 07:39:11.041855 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:11.041769 2573 scope.go:117] "RemoveContainer" containerID="704a58da1fbd74633b11b096eca67f98519fe581993f5a7c87463d9439da7e74" Jul 07 07:39:11.042503 ip-10-0-131-241 kubenswrapper[2573]: E0707 07:39:11.042473 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"704a58da1fbd74633b11b096eca67f98519fe581993f5a7c87463d9439da7e74\": container with ID starting with 704a58da1fbd74633b11b096eca67f98519fe581993f5a7c87463d9439da7e74 not found: ID does not exist" containerID="704a58da1fbd74633b11b096eca67f98519fe581993f5a7c87463d9439da7e74" Jul 07 07:39:11.042625 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:11.042512 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"704a58da1fbd74633b11b096eca67f98519fe581993f5a7c87463d9439da7e74"} err="failed to get container status \"704a58da1fbd74633b11b096eca67f98519fe581993f5a7c87463d9439da7e74\": rpc error: code = NotFound desc = could not find container \"704a58da1fbd74633b11b096eca67f98519fe581993f5a7c87463d9439da7e74\": container with ID starting with 704a58da1fbd74633b11b096eca67f98519fe581993f5a7c87463d9439da7e74 not found: ID does not exist" Jul 07 07:39:11.082989 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:11.082957 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b"] Jul 07 07:39:11.089376 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:11.089354 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1082ad-6d8b999cdd-z4p7b"] Jul 07 07:39:11.109047 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:11.109019 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc357ad5-5faf-4af3-8744-91fdca5729c9" path="/var/lib/kubelet/pods/dc357ad5-5faf-4af3-8744-91fdca5729c9/volumes" Jul 07 07:39:18.342149 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:18.342117 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:39:18.461802 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:18.461772 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:39:18.717903 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:18.717809 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:39:18.936501 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:18.936469 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:39:24.675657 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:24.675621 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-twmpt_e1b1f08e-b37d-4cec-a942-87795b69dfec/manager/0.log" Jul 07 07:39:24.812063 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:24.812031 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gdc4p_0b0f2bfd-706a-46ec-a92e-a02c5afe9b15/manager/0.log" Jul 07 07:39:24.958106 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:24.958020 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-c87c98569-w4sbj_557b6ddd-adcf-4000-9175-a90c34dd4a37/maas-api/0.log" Jul 07 07:39:25.386038 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:25.386009 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c8d7dbb94-2jrqx_906aa68f-c3b7-498b-8dae-4298deca39bd/manager/0.log" Jul 07 07:39:25.523304 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:25.523275 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gdsdv_9b123dbe-7add-499e-a759-0bd4537e13e8/server/0.log" Jul 07 07:39:25.658282 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:25.658205 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2j7jp_61c48a2b-f03d-4c37-9872-32773d868d00/manager/0.log" Jul 07 07:39:26.104676 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:26.104646 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbf9cc589-xr7bj_06c685c0-2e46-42be-b4c2-e4b65210432e/copy-manifests/0.log" Jul 07 07:39:26.123485 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:26.123459 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbf9cc589-xr7bj_06c685c0-2e46-42be-b4c2-e4b65210432e/manager/0.log" Jul 07 07:39:26.268058 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:26.268033 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-7hlst_832f39b3-d091-4a25-b80a-241d675b5029/postgres/0.log" Jul 07 07:39:27.887377 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:27.887286 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-cqx5c_d5abeee0-d813-4ea1-819f-0016c04901f7/manager/0.log" Jul 07 07:39:29.119612 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:29.119583 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-l6hdh_c8987396-97f4-4963-86e5-3634adc1aec4/discovery/0.log" Jul 07 07:39:29.347614 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:29.347588 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54bd8c98d7-rj6wt_895e7b8b-350b-439b-abfd-351702707f22/kube-auth-proxy/0.log" Jul 07 07:39:29.626163 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:29.626134 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-44lzc_2cb8b73d-18bb-4ca8-b2af-a00c971c928d/payload-pre-processing/0.log" Jul 07 07:39:29.748537 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:29.748506 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-kk64t_5c1d716f-4fcd-4bf9-bda4-19db7059dc37/payload-processing/0.log" Jul 07 07:39:30.228856 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:30.228828 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-4mmlb_b85b4e24-079d-48d5-bf0b-44a35d609034/main/0.log" Jul 07 07:39:30.599611 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:30.599581 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrp6n4_848a836a-7ed4-4e10-9430-66c0fe1a645d/main/0.log" Jul 07 07:39:30.726240 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:30.726214 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-9pj6w_a11a1121-6fe9-4579-87f2-344f019c2585/main/0.log" Jul 07 07:39:35.115692 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:35.115660 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:39:35.284096 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:35.284063 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:39:39.003875 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:39.003843 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6zrvs_3fde7f49-f438-445e-bfc1-d645d3157b33/global-pull-secret-syncer/0.log" Jul 07 07:39:39.256175 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:39.256094 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" Jul 07 07:39:39.497190 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:39.497165 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-241.ec2.internal_c079fdaed824bcba1a1cdfbf52df0d2c/haproxy/0.log" Jul 07 07:39:44.361793 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:44.361766 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-cqx5c_d5abeee0-d813-4ea1-819f-0016c04901f7/manager/0.log" Jul 07 07:39:46.858970 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:46.858939 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-2xsrx_27f2c73a-c551-4fe0-9563-00c9cd765437/cluster-monitoring-operator/0.log" Jul 07 07:39:47.483409 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:47.483379 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vp4gn_318f8939-2324-4c3b-9428-9e16f7487771/node-exporter/0.log" Jul 07 07:39:47.516406 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:47.516376 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vp4gn_318f8939-2324-4c3b-9428-9e16f7487771/kube-rbac-proxy/0.log" Jul 07 07:39:47.549446 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:47.549421 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vp4gn_318f8939-2324-4c3b-9428-9e16f7487771/init-textfile/0.log" Jul 07 07:39:48.484515 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.484484 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst"] Jul 07 07:39:48.484884 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.484841 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="849545ba-ee76-4812-b2cc-bbe1f2337ffc" containerName="maas-api" Jul 07 07:39:48.484884 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.484854 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="849545ba-ee76-4812-b2cc-bbe1f2337ffc" containerName="maas-api" Jul 07 07:39:48.484884 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.484867 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc357ad5-5faf-4af3-8744-91fdca5729c9" containerName="maas-api" Jul 07 07:39:48.484884 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.484873 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc357ad5-5faf-4af3-8744-91fdca5729c9" containerName="maas-api" Jul 07 07:39:48.485015 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.484931 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="849545ba-ee76-4812-b2cc-bbe1f2337ffc" containerName="maas-api" Jul 07 07:39:48.485015 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.484941 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc357ad5-5faf-4af3-8744-91fdca5729c9" containerName="maas-api" Jul 07 07:39:48.487670 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.487654 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.490981 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.490959 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zwr9g\"/\"default-dockercfg-r2qnp\"" Jul 07 07:39:48.490981 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.490978 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zwr9g\"/\"openshift-service-ca.crt\"" Jul 07 07:39:48.491608 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.491593 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zwr9g\"/\"kube-root-ca.crt\"" Jul 07 07:39:48.503605 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.503585 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst"] Jul 07 07:39:48.558943 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.558910 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkhn8\" (UniqueName: \"kubernetes.io/projected/b4b0daf3-b5e3-496d-98ab-03f5f9b79919-kube-api-access-hkhn8\") pod \"perf-node-gather-daemonset-jsfst\" (UID: \"b4b0daf3-b5e3-496d-98ab-03f5f9b79919\") " pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.559098 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.558965 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b4b0daf3-b5e3-496d-98ab-03f5f9b79919-lib-modules\") pod \"perf-node-gather-daemonset-jsfst\" (UID: \"b4b0daf3-b5e3-496d-98ab-03f5f9b79919\") " pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.559098 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.559017 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4b0daf3-b5e3-496d-98ab-03f5f9b79919-sys\") pod \"perf-node-gather-daemonset-jsfst\" (UID: \"b4b0daf3-b5e3-496d-98ab-03f5f9b79919\") " pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.559098 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.559053 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b4b0daf3-b5e3-496d-98ab-03f5f9b79919-podres\") pod \"perf-node-gather-daemonset-jsfst\" (UID: \"b4b0daf3-b5e3-496d-98ab-03f5f9b79919\") " pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.559098 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.559092 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b4b0daf3-b5e3-496d-98ab-03f5f9b79919-proc\") pod \"perf-node-gather-daemonset-jsfst\" (UID: \"b4b0daf3-b5e3-496d-98ab-03f5f9b79919\") " pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.660255 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.660223 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkhn8\" (UniqueName: \"kubernetes.io/projected/b4b0daf3-b5e3-496d-98ab-03f5f9b79919-kube-api-access-hkhn8\") pod \"perf-node-gather-daemonset-jsfst\" (UID: \"b4b0daf3-b5e3-496d-98ab-03f5f9b79919\") " pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.660451 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.660292 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b4b0daf3-b5e3-496d-98ab-03f5f9b79919-lib-modules\") pod \"perf-node-gather-daemonset-jsfst\" (UID: \"b4b0daf3-b5e3-496d-98ab-03f5f9b79919\") " pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.660451 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.660317 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4b0daf3-b5e3-496d-98ab-03f5f9b79919-sys\") pod \"perf-node-gather-daemonset-jsfst\" (UID: \"b4b0daf3-b5e3-496d-98ab-03f5f9b79919\") " pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.660451 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.660374 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b4b0daf3-b5e3-496d-98ab-03f5f9b79919-podres\") pod \"perf-node-gather-daemonset-jsfst\" (UID: \"b4b0daf3-b5e3-496d-98ab-03f5f9b79919\") " pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.660451 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.660401 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b4b0daf3-b5e3-496d-98ab-03f5f9b79919-proc\") pod \"perf-node-gather-daemonset-jsfst\" (UID: \"b4b0daf3-b5e3-496d-98ab-03f5f9b79919\") " pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.660633 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.660495 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4b0daf3-b5e3-496d-98ab-03f5f9b79919-sys\") pod \"perf-node-gather-daemonset-jsfst\" (UID: \"b4b0daf3-b5e3-496d-98ab-03f5f9b79919\") " pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.660633 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.660492 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b4b0daf3-b5e3-496d-98ab-03f5f9b79919-lib-modules\") pod \"perf-node-gather-daemonset-jsfst\" (UID: \"b4b0daf3-b5e3-496d-98ab-03f5f9b79919\") " pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.660633 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.660531 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b4b0daf3-b5e3-496d-98ab-03f5f9b79919-podres\") pod \"perf-node-gather-daemonset-jsfst\" (UID: \"b4b0daf3-b5e3-496d-98ab-03f5f9b79919\") " pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.660633 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.660528 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b4b0daf3-b5e3-496d-98ab-03f5f9b79919-proc\") pod \"perf-node-gather-daemonset-jsfst\" (UID: \"b4b0daf3-b5e3-496d-98ab-03f5f9b79919\") " pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.683620 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.683593 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkhn8\" (UniqueName: \"kubernetes.io/projected/b4b0daf3-b5e3-496d-98ab-03f5f9b79919-kube-api-access-hkhn8\") pod \"perf-node-gather-daemonset-jsfst\" (UID: \"b4b0daf3-b5e3-496d-98ab-03f5f9b79919\") " pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.798238 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.798208 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:48.938290 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:48.938263 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst"] Jul 07 07:39:48.940940 ip-10-0-131-241 kubenswrapper[2573]: W0707 07:39:48.940914 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb4b0daf3_b5e3_496d_98ab_03f5f9b79919.slice/crio-8fc6e387860843a963565c5dc41273cd76a9997698db1cc7c814b8889eb5fb57 WatchSource:0}: Error finding container 8fc6e387860843a963565c5dc41273cd76a9997698db1cc7c814b8889eb5fb57: Status 404 returned error can't find the container with id 8fc6e387860843a963565c5dc41273cd76a9997698db1cc7c814b8889eb5fb57 Jul 07 07:39:49.211171 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:49.211080 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" event={"ID":"b4b0daf3-b5e3-496d-98ab-03f5f9b79919","Type":"ContainerStarted","Data":"9c1d086d65d4e63ccca4ba6c624e0f59df20c24ec89e3bf2a7287633ef3e8311"} Jul 07 07:39:49.211171 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:49.211130 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" event={"ID":"b4b0daf3-b5e3-496d-98ab-03f5f9b79919","Type":"ContainerStarted","Data":"8fc6e387860843a963565c5dc41273cd76a9997698db1cc7c814b8889eb5fb57"} Jul 07 07:39:49.211407 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:49.211195 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:49.237640 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:49.237586 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" podStartSLOduration=1.23757019 podStartE2EDuration="1.23757019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 07:39:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:39:49.236106875 +0000 UTC m=+2444.572073192" watchObservedRunningTime="2026-07-07 07:39:49.23757019 +0000 UTC m=+2444.573536516" Jul 07 07:39:49.531246 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:49.531219 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-4g8xw_19033e5d-d309-40af-a5dd-99322d4c8bd3/networking-console-plugin/0.log" Jul 07 07:39:50.188958 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:50.188928 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/2.log" Jul 07 07:39:50.192967 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:50.192950 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ndp4z_08be047a-e24d-4b3a-868f-45d487a6c066/console-operator/1.log" Jul 07 07:39:52.085838 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:52.085807 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4stvb_2ec89b25-d95f-4f37-8486-9e6b81cdbaa7/konnectivity-agent/0.log" limit=1024 Jul 07 07:39:52.946165 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:52.946138 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pwjq8_a63477af-964c-4ef4-a12c-fa205ac66c9d/dns/0.log" Jul 07 07:39:52.979709 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:52.979688 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pwjq8_a63477af-964c-4ef4-a12c-fa205ac66c9d/kube-rbac-proxy/0.log" Jul 07 07:39:53.013626 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:53.013607 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2g66m_a29da4ef-3d23-4022-bd07-dd84bae55b5e/dns-node-resolver/0.log" Jul 07 07:39:53.765698 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:53.765670 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bm5dq_0c622a1f-4a29-45b7-a0de-785b67e27a5c/node-ca/0.log" Jul 07 07:39:55.083603 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:55.083566 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-l6hdh_c8987396-97f4-4963-86e5-3634adc1aec4/discovery/0.log" Jul 07 07:39:55.141447 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:55.141419 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54bd8c98d7-rj6wt_895e7b8b-350b-439b-abfd-351702707f22/kube-auth-proxy/0.log" Jul 07 07:39:55.225804 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:55.225775 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zwr9g/perf-node-gather-daemonset-jsfst" Jul 07 07:39:55.284260 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:55.284230 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-44lzc_2cb8b73d-18bb-4ca8-b2af-a00c971c928d/payload-pre-processing/0.log" Jul 07 07:39:55.345430 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:55.345357 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-kk64t_5c1d716f-4fcd-4bf9-bda4-19db7059dc37/payload-processing/0.log" Jul 07 07:39:56.264757 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:56.264723 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dw7c2_9e8f7127-1872-4e43-a37d-d73d7b235a58/serve-healthcheck-canary/0.log" Jul 07 07:39:56.884797 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:56.884766 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-j2872_b490dc3b-4f35-4db0-be34-cc804b50c62f/insights-operator/0.log" Jul 07 07:39:56.885140 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:56.885125 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-j2872_b490dc3b-4f35-4db0-be34-cc804b50c62f/insights-operator/1.log" Jul 07 07:39:56.922425 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:56.922399 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9wst6_9990b17f-cccd-4d39-a280-f179cf67e9c9/kube-rbac-proxy/0.log" Jul 07 07:39:56.956140 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:56.956110 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9wst6_9990b17f-cccd-4d39-a280-f179cf67e9c9/exporter/0.log" Jul 07 07:39:56.991714 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:56.991691 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9wst6_9990b17f-cccd-4d39-a280-f179cf67e9c9/extractor/0.log" Jul 07 07:39:59.119900 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:59.119819 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-l6hdh_c8987396-97f4-4963-86e5-3634adc1aec4/discovery/0.log" Jul 07 07:39:59.158163 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:59.158133 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54bd8c98d7-rj6wt_895e7b8b-350b-439b-abfd-351702707f22/kube-auth-proxy/0.log" Jul 07 07:39:59.274808 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:59.274782 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-44lzc_2cb8b73d-18bb-4ca8-b2af-a00c971c928d/payload-pre-processing/0.log" Jul 07 07:39:59.320223 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:59.320194 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-kk64t_5c1d716f-4fcd-4bf9-bda4-19db7059dc37/payload-processing/0.log" Jul 07 07:39:59.910225 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:39:59.910146 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-twmpt_e1b1f08e-b37d-4cec-a942-87795b69dfec/manager/0.log" Jul 07 07:40:00.050957 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:40:00.050930 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gdc4p_0b0f2bfd-706a-46ec-a92e-a02c5afe9b15/manager/0.log" Jul 07 07:40:00.092791 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:40:00.092764 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-c87c98569-w4sbj_557b6ddd-adcf-4000-9175-a90c34dd4a37/maas-api/0.log" Jul 07 07:40:00.251719 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:40:00.251693 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c8d7dbb94-2jrqx_906aa68f-c3b7-498b-8dae-4298deca39bd/manager/0.log" Jul 07 07:40:00.281853 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:40:00.281827 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gdsdv_9b123dbe-7add-499e-a759-0bd4537e13e8/server/0.log" Jul 07 07:40:00.357356 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:40:00.357310 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2j7jp_61c48a2b-f03d-4c37-9872-32773d868d00/manager/0.log" Jul 07 07:40:00.549781 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:40:00.549697 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbf9cc589-xr7bj_06c685c0-2e46-42be-b4c2-e4b65210432e/manager/0.log" Jul 07 07:40:00.571404 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:40:00.571378 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbf9cc589-xr7bj_06c685c0-2e46-42be-b4c2-e4b65210432e/copy-manifests/0.log" Jul 07 07:40:00.600737 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:40:00.600711 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-7hlst_832f39b3-d091-4a25-b80a-241d675b5029/postgres/0.log" Jul 07 07:40:00.667047 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:40:00.667021 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-l6hdh_c8987396-97f4-4963-86e5-3634adc1aec4/discovery/0.log" Jul 07 07:40:00.701879 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:40:00.701854 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54bd8c98d7-rj6wt_895e7b8b-350b-439b-abfd-351702707f22/kube-auth-proxy/0.log" Jul 07 07:40:00.814426 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:40:00.814344 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-44lzc_2cb8b73d-18bb-4ca8-b2af-a00c971c928d/payload-pre-processing/0.log" Jul 07 07:40:00.854087 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:40:00.854053 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-kk64t_5c1d716f-4fcd-4bf9-bda4-19db7059dc37/payload-processing/0.log" Jul 07 07:40:02.071828 ip-10-0-131-241 kubenswrapper[2573]: I0707 07:40:02.071800 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-87nb4_2fe61871-dda9-44d2-8076-8a9022838ea9/openshift-lws-operator/0.log"